We believe, a solid, performant, and reliable gateway lays the foundation to help build the next level of LLM apps. It decreases excessive reliance on any one company and takes the focus back to building instead of spending time fixing the nitty gritties of different providers and making them work together.

Features:

  Blazing fast (9.9x faster) with a tiny footprint (~45kb installed)
  Load balance across multiple models, providers, and keys
  Fallbacks make sure your app stays resilient
  Automatic Retries with exponential fallbacks come by default
  Plug-in middleware as needed
  Battle tested over 100B tokens and millions of requests

Reach out to us at [email protected] or explore the project: https://github.com/portkey-ai/gateway