RAG Without a Vector DB, PostgreSQL and Faiss for AI-Powered Docs

4 points

a year ago

We've built Doclink.io, an AI-powered document analysis product with a from-scratch RAG implementation that uses PostgreSQL for persistent, high-performance storage of embeddings and document structure. Most RAG implementations today rely on vector databases for document chunking, but they often lack customization options and can become costly at scale. Instead, we used a different approach: storing every sentence as an embedding in PostgreSQL. This gave us more control over retrieval while allowing us to manage both user-related and document-related data in a single SQL database.

At first, with a very basic RAG implementation, our answer relevancy was only 45%. We read every RAG related paper and try to get best practice methods to increase accuracy. We tested and implemented methods such as HyDE (Hypothetical Document Embeddings), header boosting, and hierarchical retrieval to improve accuracy to over 90%.

One of the biggest challenges was maintaining document structure during retrieval. Instead of retrieving arbitrary chunks, we use SQL joins to reconstruct the hierarchical context, connecting sentences to their parent headers. This ensures that the LLM receives properly structured information, reducing hallucinations and improving response accuracy.

Since we had no prior web development experience, we decided to build a simple Python backend with a JS frontend and deploy it on a VPS. You can use the product completely for free. We have a one time payment premium plan for lifetime, but this plan is for the users want to use it excessively. Mostly you can go with the free plan.

If you're interested in the technical details, we're fully open-source. You can see the technical implementation in GitHub (https://github.com/rahmansahinler1/doclink) or try it at doclink.io

Would love to hear from others who have explored RAG implementations or have ideas for further optimization!

2 comments