The challenge#
A company's internal knowledge base typically exists in scattered form: in Google Docs, PDFs, email threads, Slack messages, old Notion pages. When a customer asks a question, the team has to spend minutes — sometimes tens of minutes — searching for the answer. The problem doesn't improve as the company grows; it gets exponentially worse.
This is an especially sore point for 10-50 person businesses. There's no dedicated knowledge management team, no enterprise-grade internal wiki, and informal knowledge transfer — the "ask Steve, he knows" culture — is the primary method. This works as long as Steve is available. When he's on vacation, sick, or working on something else, the entire team is stuck.
This creates several reinforcing problems:
- Slow response time: The customer waits while the colleague searches through three different systems. Sometimes they never find the answer and try to recall from memory
- Inconsistent answers: Two colleagues find different versions of the same information, causing trust erosion with the customer
- Knowledge loss: When an experienced colleague leaves, the knowledge in their head leaves with them. It can take weeks or months for a new hire to fill that gap
- Doesn't scale: As the team and documentation grow, search gets slower. What's a 2-minute search at 5 people becomes daily frustration at 30
- Onboarding cost: Every new colleague's training means answering the same questions over and over again — pulling senior team members away from productive work
The goal: a chatbot that answers from the company's own documents — accurately, quickly, and with source references.
RAG (Retrieval-Augmented Generation) is an AI technique that combines search with text generation. The core concept is that we don't use the AI model on its own, but first find the relevant information, and only then ask the model to formulate an answer from it.
The process step by step:
- The user asks a question in natural language, exactly as they'd ask a colleague
- The system finds the relevant document segments from a vector database — this is the "retrieval" part. The search isn't keyword-based but semantic: the system compares the meaning of the question with the meaning of the documents
- The AI formulates the answer based on the found context — this is the "generation" part. The model can only use information returned by the search
- The answer includes the source reference so it can be verified. The user can always look up the original document
The key point: the AI doesn't answer from its own (sometimes incorrect) knowledge, but exclusively from the provided documents. If there's no relevant information, it says: "I couldn't find an answer to this question in the knowledge base." This behavior is built in — the system would rather not answer than make something up.
This is the biggest advantage over ChatGPT: the RAG chatbot doesn't hallucinate, because it doesn't invent answers. While ChatGPT generates answers from the general knowledge it learned on the internet (which may be accurate, but may also be completely wrong), the RAG chatbot relies only on your documents.
The solution in detail#
Document import and preparation
We collected and processed the company's documents. This step is critical because the chatbot's quality directly depends on the documentation quality. The system handles the following formats:
- PDF documents (manuals, procedures, contract templates)
- Google Docs and Notion pages (synced via API, so updates are automatically included)
- Markdown files (technical documentation, developer guides)
- Plain text files (FAQ lists, email templates, product descriptions)
We split the documents into intelligent segments: not line by line, but by content units. A chapter, a question-answer pair, a procedure description — these become the search units. The quality of segmentation directly affects answer accuracy, so we tune this individually for every project.
Vectorization — the foundation of search
Every segment is converted to a vector by the OpenAI embedding model and stored in Supabase pgvector. A vector is a mathematical representation that describes the meaning of the text — not just the words, but the intent. A 1536-dimensional number sequence that captures the semantic content of the text.
Thanks to this, search is semantic: if the user asks "how do we add a new user?", the system finds the "User Management → Create New User" section, even if the exact words don't match. The pgvector cosine similarity algorithm ranks results by relevance, so the most fitting segments always form the basis of the answer.
Question processing — under 3 seconds
When the user asks a question, the system runs through the following pipeline, fully automated:
- The question is converted to a vector (using the same embedding model as the documents)
- pgvector finds the closest document segments (top 3-5 relevant results)
- The segments are given to the OpenAI GPT model as context, along with a carefully crafted system prompt
- The model formulates a natural language answer based on the context, taking language-specific nuances into account
- The answer includes the source document name and section, so anyone can verify it
The entire process takes < 3 seconds, even for complex questions. Response time mostly depends on the OpenAI API call — the vector search itself runs in milliseconds.
Source attribution — trust and verification
Every answer includes the source reference: which document and which section the information comes from. The user can view the original document with a single click, verifying the chatbot's information at any time.
This is critical for trust: the team knows the chatbot's answers are verifiable, rather than the chatbot having "opinions." Source attribution is also useful for discovering when a document is outdated — if the chatbot cites an old version of a procedure, that's an immediate signal that it needs updating. In this way, the chatbot indirectly encourages documentation maintenance.
Before and after#
- Finding answers: 3-10 min of manual searching
- Inconsistent: two colleagues give different answers
- Only long-tenured staff know the answers
- Documentation is outdated, nobody updates it
- New colleague onboarding: takes weeks
- < 3 sec AI answer with source reference
- Consistent: always from the latest document
- Anyone can ask, 0 prior knowledge needed
- The chatbot motivates documentation updates
- New colleague productive within days
RAG Chatbot — Chat interface
User: How do I add a new client to the CRM?
AI answer: To add a new client, follow these steps:
- Open the Airtable CRM view
- Click the "+ New row" button
- Fill in the required fields: company name, contact name, email, phone
- Select the status: "New lead"
- After saving, the system automatically sends a welcome email
📄 Source: Client Management Handbook > Adding a New Client (Chapter 3)
The RAG chatbot isn't ideal for every situation. It's important to understand its limitations clearly before implementing:
- It doesn't replace complex decision-making. If the answer to a question is "it depends," the chatbot can't weigh the circumstances. For example, a quote calculation where the price depends on quantity, deadline, and client status is too complex for RAG
- It's only as good as the documentation. If the documentation is outdated or incomplete, the chatbot's answers will be too. If the shipping policy hasn't been updated since last year, the chatbot will cite the old terms — confidently
- It doesn't handle confidential information automatically. If the documentation contains internal pricing, HR data, or payroll, access control needs to be handled separately. By default, every user has access to every document — permission levels must be configured separately
- It doesn't learn from conversations. If a user corrects the chatbot, it doesn't automatically update the documentation — that must be done manually. The chatbot doesn't get "smarter" from usage, only from documentation updates
- It doesn't handle multi-step processes. If the answer depends on where the user is in a process (e.g., an approval workflow), the chatbot can't track state — it handles every question independently
- Language limitations. If documentation is mixed-language (e.g., a mix of Hungarian and English technical texts), search accuracy may decrease because semantic similarity works best within a single language
RAG is best for: frequent, repetitive questions where the answer can clearly be found in the documentation and speed is critical. Think of the 80% of customer service questions — the ones the team has already answered a hundred times.
Results in numbers#
| Metric | Before | After |
|---|---|---|
| Response time (customer service) | 3-10 min | < 3 seconds |
| Consistency | Variable (colleague-dependent) | 100% (document-based) |
| New colleague onboarding | 2-4 weeks | 3-5 days |
| Knowledge loss risk | High (key person dependent) | Low (documented) |
Customer service response time dropped to a fraction. The chatbot answers the most common questions instantly, and passes more complex cases to the team with the appropriate context.
How to apply this in your business#
You don't need to load the entire company knowledge base at once. The best approach is gradual, and it delivers measurable results in the first week:
-
Map out the most frequent questions. Ask your team to note down for a week what questions they look up answers for. Usually 10-15 topic areas cover 80% of the questions — those need documentation first.
-
Gather the basic documents. Start with 10-20 documents: procedures, product descriptions, FAQs, onboarding guides. If there are no formal documents, ask senior colleagues to dictate the most important information — even as audio recordings that we then convert to text.
-
Test with a small group. Give access to 3-5 colleagues for a two-week trial period. Ask them to try the chatbot first for every internal question, and to flag if the answer is incomplete or inaccurate. This doubles as a documentation quality audit.
-
Iterate based on feedback. The trial period feedback reveals where documentation is incomplete. Fill in the gaps, fine-tune the segmentation, and add new documents for the topics that came up.
-
Expand gradually. If the chatbot works well with the basic documentation, add more areas: financial processes, HR policies, technical documentation. Vectorization takes a few minutes per document — expansion doesn't require downtime.
Our experience: most teams by the second week are already asking to load more documents, because they've experienced how much time they save. The chatbot also motivates documentation updates — if the team sees they're getting outdated information back, they update the original document faster.
If you'd like a RAG chatbot for your company, book a free consultation.
Tech stack#
| Tool | Role |
|---|---|
| n8n | Document import workflow, question-answer pipeline |
| OpenAI API | Embedding (vectorization) and answer generation |
| Supabase | PostgreSQL database + pgvector extension |
| pgvector | Vector search across document segments |