README.md 4.01 KB
Newer Older
ale's avatar
ale committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
usermetadb
==========

The *User Metadata Database* (`usermetadb`) stores long-term information
about user access patterns in order to detect anomalous behavior and
implement other safety checks. It strives to do so while respecting
the anonymity of the users, focusing on information that is actually
useful to them.

In practical terms, it stores the following information on every
successful login:
* *timestamps*, quantized/fuzzed to a sufficiently large amount (1h?)
  to make correlation difficult
* *location*, stored at the country level based on user IP
* *device information* based on long-term cookies

The idea is that this is enough information to provide users with
meaningful summaries such as "you have just logged in from a
new/unknown device" and "you logged in earlier today with Chrome on a
mobile Android device", without storing de-anonymizing information on
the server side.

The cookie-based device detection might present an issue from this
point of view, because it allows to establish a forensic link between
a specific device and an account if one is in possession of the
server-side log database (only partially mitigated by the fact that
the cookie is encrypted).

shammash's avatar
shammash committed
29 30 31 32 33
`usermetadb` also stores last-login information for internal infrastructure
maintenance. The idea is to retain the minimal amount of information to perform
tasks such as "disable accounts that have not been active for more than N
years".

ale's avatar
ale committed
34
# API
ale's avatar
ale committed
35 36 37 38 39 40 41 42 43

The server exports an API over HTTP/HTTPS, all requests should be made
using the POST method and an *application/json* Content-Type. The
request body should contain a JSON-encoded request object. Responses
will similarly be JSON-encoded.

The API is split into two conceptually separate sets, the *log* API
and the *analysis* API.

ale's avatar
ale committed
44
## Log API
ale's avatar
ale committed
45 46

`/api/add_log` (*AddLogRequest*)
shammash's avatar
shammash committed
47

ale's avatar
ale committed
48 49 50 51
Stores a new log entry for a user in the database. The request must be
a `LogEntry` object. The method returns an empty response.  If the log
entry contains device information, the list of devices for the
specified user is updated with that information.
ale's avatar
ale committed
52 53

`/api/get_user_logs` (*GetUserLogsRequest*) -> *GetUserLogsResponse*
shammash's avatar
shammash committed
54

ale's avatar
ale committed
55
Returns recent logs for a specific user.
ale's avatar
ale committed
56 57 58

`/api/get_user_devices` (*GetUserDevicesRequest*) -> *GetUserDevicesResponse*

ale's avatar
ale committed
59
Returns the list of known devices for a user.
ale's avatar
ale committed
60 61 62 63 64

## Analysis API

`/api/check_device` (*CheckDeviceRequest*) -> *CheckDeviceResponse*

ale's avatar
ale committed
65 66
Returns information about a device, whether we have seen it before, if
the localization information matches the historical trend, etc.
ale's avatar
ale committed
67

shammash's avatar
shammash committed
68 69 70 71 72 73 74 75 76 77 78 79 80 81
## Last-login API

`/api/set_last_login` (*SetLastLoginRequest*)

Stores the last login of a user in the database. The request must be a
`LastLoginEntry` object. The method returns an empty response. The service name
must be specified in the last login entry.

`/api/get_last_login` (*GetLastLoginRequest*) -> *GetLastLoginResponse*

Returns the last login of a given user. If the service name is specified it
returns the last login for that specific service, otherwise return last login
for all services.

ale's avatar
ale committed
82 83 84
`/api/get_unused_accounts` (*GetUnusedAccountsRequest*) -> *GetUnusedAccountsResponse*

Returns accounts that have not been used in a specified amount of time.
ale's avatar
ale committed
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105

# Configuration

The configuration is contained in a YAML-encoded file, normally
`/etc/user-meta-server.yml` (this can be changed using the
*--config* command-line flag).

The known attributes are:

* `db_uri` is the database URI. As currently only the *sqlite3* driver
  is supported, this is just the path to the database file.
* `http_server` specifies standard parameters for the HTTP server:
  * `tls` contains the server-side TLS configuration:
    * `cert` is the path to the server certificate
    * `key` is the path to the server's private key
    * `ca` is the path to the CA used to validate clients
    * `acl` specifies TLS-based access controls, a list of entries
      with the following attributes:
      * `path` is a regular expression to match the request URL path
      * `cn` is a regular expression that must match the CommonName
        part of the subject of the client certificate