reports-collector
Proof of concept of an ingestion service for anomaly reports.
Supports receiving reports over HTTPS and SMTP for the following mechanisms:
The reports are logged in structured denormalized form to standard output, the idea being that a log collection pipeline would be in place to get them into Elasticsearch or some other index.
The source IP address is not included in the output, but for user reports we add instead the ASN, to allow for some meaningful non-deanonymizing aggregation of the reports.
The SMTP receiver works by running a very simple SMTP server (on the port specified by --smtp-addr), meant to be downstream to your MTA. This SMTP server does not check the recipient address and just attempts to parse all received email.
Things that are still to do:
- Implement email ingestion support for TLS-RPT reports
- More testing with real-world reports
Right now the server offers a single intake endpoint at /ingest/v1, and then looks at the Content-Type of the request to figure out what kind of report it is. It might be just simpler to switch to separate endpoints per report type.