Show HN: Warp_cache – SIEVE cache in Rust for Python, 25x faster than cachetools

2 points

3 months ago

I built warp_cache, a thread-safe Python caching decorator backed by a Rust extension (PyO3). It's designed as a drop-in replacement for functools.lru_cache.

The main ideas:

- SIEVE eviction (NSDI'24) instead of LRU — scan-resistant, up to 21.6% fewer cache misses on real workloads - The entire cache lookup happens in a single Rust __call__ — no Python wrapper overhead - Thread-safe out of the box: GilCell under the GIL, sharded RwLock under free-threaded Python (3.13+) - Cross-process shared memory backend via mmap (9.7M ops/s across processes) - 16–23M ops/s single-threaded, 25x faster than cachetools, 1.6x faster than lru_cache + Lock under multi-threaded load

Migration from lru_cache is one line:

  -from functools import lru_cache
  +from warp_cache import cache

  -@lru_cache(maxsize=128)
  +@cache(max_size=128)

Benchmarks and eviction quality comparisons are in the repo.