My name is Dillion and I'm the creator of llm.report.

A few months ago, I was frustrated by the lack of observability into the OpenAI API.

All of us are left in the dark about API performance, latency, cost calculation, cost breakdown, and more. I just wanted to know more about how my AI app is performing in production and make data-driven decisions to improve the product.

So I ended up just building it myself.

There are three parts to the platform:

1. OpenAI API Dashboard (no-code) - Enter your OpenAI key and get access to detailed insights straight from the OpenAI API itself

2. Logging - Log all your requests / responses to OpenAI to analyze cost, latency, token usage and more

3. User Metrics - Calculate your cost per user to figure out who your power users are, and how much they cost you per day

Would appreciate any feedback you have! Would you use something like this? Why or why not?