Show HN: Zep – Long-Term Memory Store for LLM Apps

7 points

3 years ago

Zep is a long-term memory store designed for conversational AI applications built using modern LLMs. It handles the storage, summarization, embedding, indexing, and enrichment of chat histories, and offers developers a simple, low-latency API to this data. Chat history storage is an infrastructure challenge all developers and enterprises face as they look to move from prototypes to deploying conversational AI applications that provide rich and intimate experiences to users.

Key features include long-term memory persistence, auto-summarization, vector search, auto-token counting, and Python and JavaScript SDKs. Upcoming features include Langchain Memory and Retrievers, integrations with other conversational AI frameworks, entity extraction, and much more.

Github repo: https://github.com/getzep/zep

Long-term memory persistence enables a variety of use cases, including:

- Personalized re-engagement of users based on their chat history.

- Prompt evaluation based on historical data.

- Training of new models and evaluation of existing models.

- Analysis of historical data to understand user behavior and preferences.

However:

- Most AI chat history or memory implementations run in-memory and are not designed for stateless deployments or long-term persistence.

- Standing up and managing low-latency infrastructure to store, manage, and enrich memories is non-trivial.

- When storing messages long-term, developers are exposed to privacy and regulatory obligations around PII, retention, and deletion of user data.

Zep aims to solve these challenges.

Zep and its Python and Javascript client libraries have been open-sourced under the Apache License.

Learn more and contribute:

- Zep server: https://github.com/getzep/zep

- Python SDK: https://github.com/getzep/zep-python

- Javascript SDK: https://github.com/getzep/zep-js

Daniel & Sharath

3 comments