Show HN: Improve LLM Performance by Maximizing Iteration Speed

5 points

2 years ago

LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more.

There are thousands of permutations to try. We need to be able to easily experiment with these different permutations, measure performance in an objective way, and compare performance across each other to find the best possible combination.

---

I have been working in AI since 2021 - first at FAANG with ML, then with LLM in start-ups since early 2023. I have had the chance to talk with many companies working with AI. The biggest mistake I see is a lack of standard process that allows them to rapidly iterate towards their performance goal.

Using my learnings, I’m working on an OSS framework that structures your LLM Application Development for Rapid Iteration so you can reach your performance targets much faster.

- If you are interested, you can learn more about it at: https://palico.ai/

- It's also Open Source and you can get setup with a single command. Stars are always appreciated. You can checkout the repo at: https://github.com/palico-ai/palico-main