Commit da4990e9 authored by ale's avatar ale

Initial commit

This document defines a generic API to expose user accounts and
associated resources to UIs and management tools, so that they can
delegate privileged operations to a dedicated service.
The focus is on making adding resources and functionality as simple as
possible, to support diverse service ecosystems.
We'd like to make it possible to create a generic, extensible UI for
account management, which would deal with the UX workflows, leaving
the implementation-specific details to the (custom) accountserver
service. Such application would initially support standard operations
on resources: adding functionality should require only adding a bit of
UI-related code to the application (and the implementation-specific
support on the server side).
## Data model
The accountserver data model provides a high-level representation of
user accounts and associated resources, decoupled from the underlying
database implementation (which likely consists of inter-related SQL
tables or LDAP objects). Such high-level representation is targeted at
humans, either users themselves or administrators.
The top-level object is an *account*, representing a user's identity.
Accounts contain a number of *resources* (of different types), which
it owns - meaning that it has administrative control over them.
Resources can be of different types (mailboxes, websites, mailing
lists, etc), each associated with a different service. In some cases,
resources can themselves contain other resources, to represent a close
association of some form (the semantics will be service-specific, an
example could be a website and its associated MySQL database, which
should be co-located).
Resources have a *name*, which must be globally unique for resources
of the same type. Resources within an account are uniquely identified
by an *ID*, which combines type and name and is a globally unique
<id> = <type>/<name>
The resource type `account` identifies the account itself.
Each resource type supports a number of *actions* that can modify the
state of resources. The specific actions and their semantics will need
to be documented (see [Schema](#schema) below), and need to be agreed
upon by the UIs/tools and the accountserver, but this scheme is easily
## Interface
The accountserver provides an HTTP endpoint operating as a RPC server.
Request bodies should be JSON-encoded, same as responses.
#### get(*account_name*)
Returns (public) information on an account and all the resources
contained in it.
#### action(*account_name*, *resource_id*, *action*, {*args*})
Perform a generic action on the specified resource.
### Authentication
All requests should be authenticated. We assume that there is a way to
identify certain users as *administrators* (for instance via group
membership). Action handlers in the accountserver implementation have
the option of choosing between two different ACL checks (and only
these two):
* *user*:
* allow a user access if the resource belongs to the user's own
* allow administrators access to this action for any account
* *admin-only*:
* only administrators have access to this action
In more detail, there are two options that could be implemented,
depending on the specifics of the authentication system used:
* *Trust the UI and tools*: authenticate the connection with the
accountserver, and trust the front-ends with their own internal
access control checks (so that, for instance, user x can not modify
data belonging to user y).
* *End-to-end verification*: pass an authentication token for the user
from the front-ends to the accountserver, and verify it on every
request. This puts the trust on the authentication system itself,
regardless of potential issues in the front-end implementation.
## Schema
The *schema* describes the known resource types and the actions that
can be performed on them, along with their parameters.
What follows is a sample schema, every resource type is described along with
the attributes that it supports and the actions that can be performed
on it.
### account
Represents a user account, or an *identity*. It is the primary
identificator for users in the system.
* `status`: one of *active*, *inactive*, or other states that might be
necessary to implement more complex state machines related to
account activation
* `otp_enabled`: true if OTP 2FA is enabled
* `u2f_enabled`: true if U2F 2FA is enabled
* `u2f_registrations`: list of U2F registration IDs (not the keys
themselves though)
* `set_password`: change primary account password to a new one
* `enable_otp` / `disable_otp`: enable or disable OTP 2FA
* ...
### email
### mailing list
### web hosting (website / db / FTP account or equivalent)
L'*accountserver* è il servizio che fa da interfaccia tra il database
utenti e gli altri servizi ed applicazioni interne. Disaccoppia la
struttura dei dati nel database dal concetto di *utente* ed *account*,
ed implementa la *business logic* relativa alle operazioni di alto
livello di gestione degli account.
Le motivazioni per la creazione di questo servizio includono:
* La constatazione che le applicazioni lato utente (e lato
amministratore, come *accounts*) operano su astrazioni differenti
rispetto a quanto implementato a basso livello nello schema LDAP
(utenti ed account, anziché oggetti LDAP gerarchici); inoltre, il
codice di traduzione è duplicato.
* I dettagli dell'implementazione di basso livello (LDAP) filtrano
ogni volta che scriviamo automazione che opera su utenti o account.
* La business logic delle operazioni di alto livello, come abilitare /
disabilitare un account, o cambiare una password, è spezzata
arbitrariamente tra le applicazioni front-end (come il pannello, o
accounts) e back-end (il vecchio accountserver), il che ne rende
difficile la verifica e la comprensione.
Si è preferito un servizio RPC anziché una libreria per alcuni
importanti motivi:
* Le informazioni su utenti ed account vanno aggregate da diversi
backend (LDAP, redis per dati temporanei, MySQL per noblogs, etc),
si vuole evitare una moltiplicazione dei flussi di connettività.
* Separazione dei privilegi: se tutte le operazioni di scrittura sul
database utenti sono effettuate da un unico servizio, le
applicazioni front-end possono operare con credenziali di sola
lettura (o nessuna).
* Se tutte le operazioni privilegiate sugli account vengono eseguite
da un unico processo, è più facile implementare un buon
auditing e verificarne la copertura.
# Modello dati
Il modello dati offerto da *accountserver* è piuttosto semplice:
l'oggetto di più alto livello è l'*utente*. Un utente possiede in modo
univoco un certo numero di *risorse*, che possono essere di diversi
*tipi*. Le risorse possono a loro volta avere sotto-risorse,
esprimendo una relazione di associazione di qualche tipo (come ad
esempio tra siti web e database MySQL).
Lo schema è definito esplicitamente in [types.go](types.go).
package backend
import (
ldaputil ""
// Generic interface to LDAP - allows us to stub out the LDAP client while
// testing.
type ldapConn interface {
Search(context.Context, *ldap.SearchRequest) (*ldap.SearchResult, error)
// LDAPBackend is the interface to an LDAP-backed user database.
type LDAPBackend struct {
conn ldapConn
userQuery *queryConfig
userResourceQueries []*queryConfig
// NewLDAPBackend initializes an LDAPBackend object with the given LDAP
// connection pool.
func NewLDAPBackend(pool *ldaputil.ConnectionPool, base string) *LDAPBackend {
return &LDAPBackend{
conn: pool,
userQuery: mustCompileQueryConfig(&queryConfig{
Base: "uid=${user},ou=People," + base,
Scope: "base",
userResourceQueries: []*queryConfig{
// Find all resources that are children of the main uid object.
Base: "uid=${user},ou=People," + base,
Scope: "sub",
// Find mailing lists, which are nested under a different root.
Base: "ou=Lists," + base,
Filter: "(&(objectClass=mailingList)(listOwner=${user}))",
Scope: "one",
func replaceVars(s string, vars map[string]string) string {
return os.Expand(s, func(k string) string {
return ldap.EscapeFilter(vars[k])
// queryConfig holds the parameters for a single LDAP query.
type queryConfig struct {
Base string
Filter string
Scope string
parsedScope int
func (q *queryConfig) validate() error {
if q.Base == "" {
return errors.New("empty search base")
// An empty filter is equivalent to objectClass=*.
if q.Filter == "" {
q.Filter = "(objectClass=*)"
q.parsedScope = ldap.ScopeWholeSubtree
if q.Scope != "" {
s, err := ldaputil.ParseScope(q.Scope)
if err != nil {
return err
q.parsedScope = s
return nil
func (q *queryConfig) searchRequest(vars map[string]string, attrs []string) *ldap.SearchRequest {
return ldap.NewSearchRequest(
replaceVars(q.Base, vars),
replaceVars(q.Filter, vars),
func mustCompileQueryConfig(q *queryConfig) *queryConfig {
if err := q.validate(); err != nil {
return q
func s2b(s string) bool {
switch s {
case "yes", "y", "on", "enabled", "true":
return true
return false
func newResourceFromLDAP(entry *ldap.Entry, resourceType, nameAttr string) *accountserver.Resource {
name := entry.GetAttributeValue(nameAttr)
return &accountserver.Resource{
ID: fmt.Sprintf("%s/%s", resourceType, name),
Name: name,
Type: resourceType,
Status: entry.GetAttributeValue("status"),
Shard: entry.GetAttributeValue("host"),
OriginalShard: entry.GetAttributeValue("originalHost"),
func newEmailResource(entry *ldap.Entry) (*accountserver.Resource, error) {
r := newResourceFromLDAP(entry, accountserver.ResourceTypeEmail, "mail")
r.Email = &accountserver.Email{
Aliases: entry.GetAttributeValues("mailAlternateAddr"),
Maildir: entry.GetAttributeValue("mailMessageStore"),
return r, nil
func newMailingListResource(entry *ldap.Entry) (*accountserver.Resource, error) {
r := newResourceFromLDAP(entry, accountserver.ResourceTypeMailingList, "listName")
r.List = &accountserver.MailingList{
Public: s2b(entry.GetAttributeValue("public")),
Admins: entry.GetAttributeValues("listOwner"),
return r, nil
func newWebDAVResource(entry *ldap.Entry) (*accountserver.Resource, error) {
r := newResourceFromLDAP(entry, accountserver.ResourceTypeDAV, "ftpname")
r.DAV = &accountserver.WebDAV{
Homedir: entry.GetAttributeValue("homeDirectory"),
return r, nil
func newWebsiteResource(entry *ldap.Entry) (*accountserver.Resource, error) {
var r *accountserver.Resource
if isObjectClass(entry, "subSite") {
r = newResourceFromLDAP(entry, accountserver.ResourceTypeWebsite, "alias")
r.Website = &accountserver.Website{
URL: fmt.Sprintf("https://www.%s/%s/", entry.GetAttributeValue("parentSite"), r.Name),
DisplayName: fmt.Sprintf("%s/%s", entry.GetAttributeValue("parentSite"), r.Name),
} else {
r = newResourceFromLDAP(entry, accountserver.ResourceTypeWebsite, "cn")
r.Website = &accountserver.Website{
URL: fmt.Sprintf("https://%s/", r.Name),
DisplayName: r.Name,
r.Website.Options = entry.GetAttributeValues("option")
r.Website.DocumentRoot = entry.GetAttributeValue("documentRoot")
r.Website.AcceptMail = s2b(entry.GetAttributeValue("acceptMail"))
return r, nil
func newDatabaseResource(entry *ldap.Entry) (*accountserver.Resource, error) {
r := newResourceFromLDAP(entry, accountserver.ResourceTypeDatabase, "dbname")
r.Database = &accountserver.Database{
DBUser: entry.GetAttributeValue("dbuser"),
CleartextPassword: entry.GetAttributeValue("clearPassword"),
// Databases are nested below websites, so we set the ParentID by
// looking at the LDAP DN.
if dn, err := ldap.ParseDN(entry.DN); err == nil {
parentRDN := dn.RDNs[1]
r.ParentID = fmt.Sprintf("%s/%s", accountserver.ResourceTypeWebsite, parentRDN.Attributes[0].Value)
return r, nil
func newUser(entry *ldap.Entry) (*accountserver.User, error) {
user := &accountserver.User{
Name: entry.GetAttributeValue("uid"),
Lang: entry.GetAttributeValue("preferredLanguage"),
Has2FA: (entry.GetAttributeValue("totpSecret") != ""),
//PasswordRecoveryHint: entry.GetAttributeValue("recoverQuestion"),
if user.Lang == "" {
user.Lang = "en"
return user, nil
// GetUser returns a user.
func (b *LDAPBackend) GetUser(ctx context.Context, username string) (*accountserver.User, error) {
// First of all, find the main user object, and just that one.
vars := map[string]string{"user": username}
result, err := b.conn.Search(ctx, b.userQuery.searchRequest(vars, nil))
if err != nil {
if ldap.IsErrorWithCode(err, ldap.LDAPResultNoSuchObject) {
return nil, nil
return nil, err
user, err := newUser(result.Entries[0])
if err != nil {
return nil, err
// Now run the resource queries, and accumulate results on the User
// object we just created.
// TODO: parallelize.
// TODO: add support for non-LDAP resource queries.
for _, query := range b.userResourceQueries {
result, err = b.conn.Search(ctx, query.searchRequest(vars, nil))
if err != nil {
for _, entry := range result.Entries {
// Some user-level attributes are actually stored on the email
// object, a shortcoming of the legacy A/I database model. Set
// them on the main User object.
if isObjectClass(entry, "virtualMailUser") {
user.PasswordRecoveryHint = entry.GetAttributeValue("recoverQuestion")
setAppSpecificPasswords(user, entry.GetAttributeValues("appSpecificPassword"))
// Parse the resource and add it to the User.
if r, err := parseLdapResource(entry); err == nil {
user.Resources = append(user.Resources, r)
return user, nil
func parseLdapResource(entry *ldap.Entry) (*accountserver.Resource, error) {
switch {
case isObjectClass(entry, "virtualMailUser"):
return newEmailResource(entry)
case isObjectClass(entry, "ftpAccount"):
return newWebDAVResource(entry)
case isObjectClass(entry, "mailingList"):
return newMailingListResource(entry)
case isObjectClass(entry, "dbMysql"):
return newDatabaseResource(entry)
case isObjectClass(entry, "subSite") || isObjectClass(entry, "virtualHost"):
return newWebsiteResource(entry)
return nil, errors.New("unknown LDAP resource")
func isObjectClass(entry *ldap.Entry, class string) bool {
classes := entry.GetAttributeValues("objectClass")
for _, c := range classes {
if c == class {
return true
return false
func parseAppSpecificPassword(asp string) (*accountserver.AppSpecificPasswordInfo, error) {
parts := strings.Split(asp, ":")
if len(parts) != 3 {
return nil, errors.New("badly encoded app-specific password")
return &accountserver.AppSpecificPasswordInfo{
Service: parts[0],
Comment: parts[2],
}, nil
func setAppSpecificPasswords(user *accountserver.User, asps []string) {
for _, asp := range asps {
if ainfo, err := parseAppSpecificPassword(asp); err == nil {
user.AppSpecificPasswords = append(user.AppSpecificPasswords, ainfo)
var siteRoot = "/home/users/"
// The hosting directory for a website is the path component immediately after
// siteRoot. This works also for sites with nested documentRoots.
func getHostingDir(path string) string {
path = strings.TrimPrefix(path, siteRoot)
if i := strings.Index(path, "/"); i > 0 {
return path[:i]
return path
// This is a very specific function meant to address a peculiar characteristic
// of the A/I legacy data model, where DAV accounts and websites do not have an
// explicit relation.
func groupWebResourcesByHomedir(resources []*accountserver.Resource) {
// Set the group name to be the 'hostingDir' for sites and DAV
// accounts. Keep a reference of websites by ID so we can later fix the
// group for databases too, via their ParentID.
webs := make(map[string]*accountserver.Resource)
for _, r := range resources {
switch r.Type {
case accountserver.ResourceTypeWebsite:
r.Group = getHostingDir(r.Website.DocumentRoot)
webs[r.ID] = r
case accountserver.ResourceTypeDAV:
r.Group = getHostingDir(r.DAV.Homedir)
// Fix databases in a second pass.
for _, r := range resources {
if r.Type == accountserver.ResourceTypeDatabase && r.ParentID != "" {
r.Group = webs[r.ParentID].Group
package main
import (
ldaputil ""
var (
addr = flag.String("addr", ":4040", "tcp `address` to listen on")
configFile = flag.String("config", "/etc/authserver/config.yml", "configuration `file`")
type Config struct {
LDAP struct {
URI string `yaml:"uri"`
BindDN string `yaml:"bind_dn"`
BindPw string `yaml:"bind_pw"`
BindPwFile string `yaml:"bind_pw_file"`
BaseDN string `yaml:"base_dn"`
} `yaml:"ldap"`
ServerConfig *serverutil.ServerConfig `yaml:"http_server"`
func (c *Config) Validate() error {
if c.LDAP.URI == "" {
return errors.New("empty ldap.uri")
if c.LDAP.BindDN == "" {
return errors.New("empty ldap.bind_dn")
if (c.LDAP.BindPwFile == "" && c.LDAP.BindPw == "") || (c.LDAP.BindPwFile != "" && c.LDAP.BindPw != "") {
return errors.New("only one of ldap.bind_pw_file or ldap.bind_pw must be set")
return nil
func loadConfig(path string) (*Config, error) {
// Read YAML config.
data, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
var config Config
if err := yaml.Unmarshal(data, &config); err != nil {
return nil, err
if err := config.Validate(); err != nil {
return nil, err
return &config, nil
func connectLDAP(config *Config) (*ldaputil.ConnectionPool, error) {
// Read the bind password.
bindPw := config.LDAP.BindPw
if config.LDAP.BindPwFile != "" {
pwData, err := ioutil.ReadFile(config.LDAP.BindPwFile)
if err != nil {
return nil, err
bindPw = strings.TrimSpace(string(pwData))
return ldaputil.NewConnectionPool(config.LDAP.URI, config.LDAP.BindDN, bindPw, 5)
func main() {
config, err := loadConfig(*configFile)
if err != nil {
pool, err := connectLDAP(config)
if err != nil {
be := backend.NewLDAPBackend(pool, config.LDAP.BaseDN)
as := server.New(be)
if err := serverutil.Serve(as.Handler(), config.ServerConfig, *addr); err != nil {
package server
import (
type Backend interface {
GetUser(context.Context, string) (*accountserver.User, error)
//GetResource(context.Context, string, string) (*accountserver.Resource, error)
type AccountServer struct {
backend Backend
func New(backend Backend) *AccountServer {
return &AccountServer{backend: backend}
func (s *AccountServer) handleGetUser(w http.ResponseWriter, r *http.Request) {
var req accountserver.GetUserRequest
if !serverutil.DecodeJSONRequest(w, r, &req) {
user, err := s.backend.GetUser(r.Context(), req.Username)
if err != nil {
log.Printf("GetUser(%s): error: %v", req.Username, err)
http.Error(w, err.Error(), http.StatusInternalServerError)
serverutil.EncodeJSONResponse(w, &accountserver.GetUserResponse{User: user})