Show HN: BerriAI – Monitor Hallucinations in LLMs (Sentry for LLM Apps)

logs.berri.ai

4 points

3 years ago

Hi HN - Ishaan and Krrish here from BerriAI. We’ve built a hallucination monitoring tool for LLM Apps in production, that can instantly identify language mistranslations (responding to a user in the incorrect language) and inventing new information errors (answering from information not in the prompt). Live demo here: https://logs.berri.ai/

We served over 1m+ chatGPT queries with our initial ‘chat with your data’ app. However, we had no ability to tell how any of the technical changes we made (e.g. moving from llama index to our own retrieval/qa system) impacted our users in production.

Berri is super easy to integrate into your system - we added it to our previous product with just 2 lines of code!

It’s super early days and we’re looking for others like us - people in production - pushing changes but unsure if/how they’re actually solving issues / improving their system over time.

Thanks for taking the time to read this, we’re really happy to be posting here :)

Krrish and Ishaan