Ask HN: Who has a smaller Redis DB with lots of reads and writes?

1 point

3 years ago

Looking for an active project or actual production deployment with thousands as opposed to millions of keys, with lots of reads and writes. I've got a way to offload most of the reads to DNS infrastructure which is very efficient for caching reads.

As far as architectural limits, no KEYS operation should generate more than roughly 64K of data and keys / values themselves should be relatively short, 64-128 bytes with a hard limit of 255 bytes per value. (Data is not restricted to e.g. ASCII, as the DNS says "any octet".)

This is a read only solution, so it implicitly protects against malicious writes; and the Redis database only has to be accessible to the translation layer. DNS features provide control over TTLs, WAF-like control over access to specific keys, restriction by IP address and "API key" (TSIG, which also provides integrity guarantees). Encryption, if necessary, can be implemented with TLS (including endpoint termination with Nginx) or VPN.

A federated architecture with multiple Redis instances is absolutely in scope, and indeed one of the "just in time collection from the edge" scenarios where the DNS particularly shines.

I'll provide 20 hours of assistance for free, what I want in return is a commitment to write up and publish a case study.

This is the tool: https://github.com/m3047/rkvdns and it would be deployed in conjunction with instances of BIND (https://www.isc.org/bind) to provide the advanced features.

Reach out to me at [email protected] or the email address referenced in the project README for paid support (bonus points for the latter) and mention you saw this post. Feel free to ask general questions below.