We're excited to open source our AI Proxy which supports Mistral, LLaMa2, OpenAI, Azure, Anthropic, and more through vanilla OpenAI SDKs. The proxy also supports configurable caching, API key management, and load balancing across multiple providers.

The proxy code also includes multiple deployment options: Cloudflare workers, Vercel, AWS Lambda, and plain-old Express. We're open sourcing this because our customers are starting to run their production workloads through it, and we believe that critical-path-of-production tools should be open source.

To play around with a hosted version of the proxy, you can simply set the base URL in your OpenAI libs to https://braintrustproxy.com/v1 (more detailed instructions in the repo[1] and our docs[2]).

We'd love your feedback -- on use cases we may not have thought of, models we should support, or other features that would be useful to include in this layer of abstraction.

[1]: https://github.com/braintrustdata/braintrust-proxy [2]: http://braintrustdata.com/docs/guides/proxy