Show HN: LlaMaKey – One master key for all cloud LLM/GenAI APIs

6 points

2 years ago

Key management for multiple users and multiple cloud LLM/GenAI APIs is difficult to be both safe and convenient. Sharing keys among users risks leaking the key and makes it difficult to curb the leakage without interruptions. But assigning one key per user per cloud API results in too many keys to keep track of.

Meet LlaMa(ster)Key, the secure and easy solution for API key management: * For each user, one master key for multiple APIs. * The master key is unique to each user. Granting and revoking a user's access won't impact other users. * The actual API keys to authenticate with cloud APIs are never known to users. Never leak a key due to a careless intern. * No code change! Keep using the official Python SDK of the cloud AI APIs, including OpenAI, Cohere, AnyScale, Perplexity, HuggingFace (free-tier inference so far), and Vectara.

LlaMaKey works by taking advantage of the BASE_URL and API_KEY/TOKEN environment variables that the official Python SDKs of major LLM/GenAI APIs rely on. Your users only need to set them via the OS environment variables or a dotenv file, or many other ways.

Yes, LlaMaKey is a proxy deployable locally or on-premises. It's totally free under the MIT license.

The Github repo is at: http://LlaMaKey.ai (will redirect to Github).

Feel free to try it out and give us feedback. You can always reach us at [email protected] or simply open an issue at Github.

2 comments