ratelimit

A collection of useful IP-based rate limit and blocker implementations in Go. By useful we mean it should offer enough features to be appropriate for handling production traffic.

Best to clear up the terminology up front:

• a rate limiter offers a one-shot API (Allow()) that acts as a simple acceptance check;
• a blocker has a more sophisticated API (Allow() and Inc()) that splits the acceptance check from receiving a signal about the event, so it can react differently according to the event status.

Some general features are shared by all implementations:

• Ability to cap the maximum amount of tracked entries (limiting total memory usage). The caller is then free to choose whether to fail open or fail close when reaching the limit.
• Masking of IP addresses to desired length (the default mask is 32 bits for IPv4 addresses, and 64 for IPv6).
• Ability to skip tracking for specific IP addresses or ranges. The default skip list only includes localhost.

The implementation focuses on two high-level types:

• RateLimiter, a simple rate limiter.
• Blocker, a rate limiter that can block high-qps peers for a specified amount of time.
• CardinalityBlocker, a more complex Blocker that looks at multiple signals beyond the simple request rate. These include a statistical estimate of the cardinality of the associated values (think username / password combinations, for instance), as well as a measure of the success/failure ratio.

The CardinalityBlocker should be able, with proper tuning, to successfully address distributed brute-forcing attempts, and distinguish them from real users proxies (e.g. CGNAT).