A/I Audit Suite

This software implements a distributed collection framework for user audit logs, which consist of high-level management events related to specific user accounts. Such logs are generated at a relatively low rate, but they are very important and must not be lost. On the other hand, this data doesn't have to be real-time: some delay in the collection pipeline is acceptable (after all, these logs are primarily useful long-term).

An interface to query the audit database is also provided, with a very simple key = value syntax.

Overview

A single audit message is a schema-less JSON dictionary describing a specific event. The meanings of its attributes are determined by convention, but a message must include at least the user, message and stamp attributes.

The design aims to be as lightweight as possible, since audit functionality has to be added to many diverse applications. It also needs to tolerate failure at any stage of the collection pipeline, to avoid dependencies on highly-available network and databases. It works as follows:

• When an audit client application wants to log a message, it tries first to deliver it to the local audit daemon (via a UNIX socket). If this fails, it will store the message in a spool directory on the local filesystem.

• The local audit daemon, which runs on every machine where audit logs are generated, is a lightweight daemon that implements the store-and-forward retry logic: when it receives an incoming message, it tries to send it to the main audit daemon, and if this fails, it stores the message in the local spool directory. It also periodically scans the spool directory and attempts to deliver messages to the main audit daemon.

• The main audit daemon receives messages and stores them in the database.

Authentication

The suite is meant to be deployed along an X509-based authentication infrastructure where a trusted Certification Authority assigns a certificate to each machine. The local audit daemons will use the machine certificates to authenticate themselves to the main audit daemon, which will consider valid any certificate signed by the CA.

Client authorization is managed using UNIX permissions: the local audit socket and spool directories are owned by the audit group, so in order to allow an application to log audit events one should simply add the user it's running as to the audit group.