Why it finally makes sense to build, not buy, a workplace search product

6 points

3 years ago

We’ve spent the last 2 months diving deep into enterprise use cases for large language models, and the one that many companies are thinking about is building a chatbot for searching through a company’s internal knowledge base.

The typical architecture goes something like this:

1. Connect to a knowledge base (Notion, Confluence, etc) and extract pages from it

2. Generate embeddings from the content and index the content based on these embeddings using a vector database like Pinecone, Weaviate, Chroma, etc

3. When someone has a query, run an approximate nearest neighbor (ANN) search on the vector database and get back ~1000 tokens of content

4. Insert the content in the context window of a prompt, and use a LLM like GPT to respond to the query based on information from the content.

A few years ago it was ridiculous to even consider building your own internal search product. Now, with libraries like LangChain (https://github.com/hwchase17/langchain), LlamaIndex (https://github.com/jerryjliu/llama_index), and open sources tools like Sidekick (https://github.com/ai-sidekick/sidekick), it’s possible to build a product that works just as well as most workplace search vendors in a matter of days, with the added benefit of being free + fully customizable. (Disclaimer: I am one of the cofounders at Sidekick)

It’s also not hard to find developers who would love to get their hands dirty with emerging technologies like vector databases, LLMs, and GPT agents. The economic incentive structure for internal search products (and maybe all SaaS product) has been flipped on its head. Why spend $100k+ year on a vendor that comes with 1-3 month implementation when you can spend half that time building something in-house that plugs in to where your team already works like Teams or Slack?

Examples: - Supabase built GPT powered search right into their docs ([https://supabase.com/docs](https://supabase.com/docs)) - PostHog built and deployed their own slackbot to answer questions about PostHog ([https://posthog.com/blog/aruba-hackathon](https://posthog.com/blog/aruba-hackathon))

Would you consider building internal semantic search in-house at your company?

5 comments