Show HN: A Go library to rate limit across millions of users

2 points

a year ago

I built a thread-safe rate limiting library for Go that's lock-free with zero allocations on the hot path. It features multiple strategies including classic token bucket and TCP-inspired AIMD algorithm for dynamic rate adjustments. Optimized for high-throughput systems with memory efficiency (packed uint64 representation) and precise per-entity limiting.

When building high-throughput services, I kept hitting the same problem with existing rate limiting libraries - they break down with high-cardinality identifiers. Try rate limiting millions of unique user IDs, API keys, or IPs and you'll face:

  1. Memory consumption exploding linearly with cardinality
  2. Lookup performance degrading as hash tables grow
  3. Contention killing throughput under load
  4. GC pressure from constant allocations

I built rate specifically to solve this:

  - Fixed memory footprint regardless of cardinality (configurable power-of-2
  buckets)
  - Sub-microsecond, zero-allocation operations even at scale
  - Lock-free design using atomic operations
  - Collision probabilities you can reason about mathematically

This means you can rate limit by user ID, device ID, API key, IP address or any other high-cardinality identifier without performance penalty. The benchmark numbers tell the story: even with thousands of distinct IDs hitting the limiter concurrently, operations complete in 1-15ns with zero allocations.