one example - I wanted to replace a marketing website with a chatbot + vector DB loaded with the previous content, docs, and blog posts. Between hallucinations, missing knowledge base info, and the LLM generally writing like an psuedo-intellectual high schooler, I realized I couldn't trust it to communicate unsupervised with my website visitors. I needed a way to improve the percent of high-quality responses, and do it this at scale.

I wired up a prototype that would

1. consult me in slack before sending any response down to a website visitor 2. incorporate my feedback into the knowledge base, and 3. reformulate answers until I approved the message 4. send it to the visitor

That prototype evolved into what is now HumanLayer https://github.com/humanlayer/humanlayer#why-humanlayer