How to Build an AI Chatbot Over Your Own Documents on Creatr

A chatbot that makes things up is worse than no chatbot. Users who catch one hallucination stop trusting every answer after it. This tutorial builds one that answers strictly from your own documents - your PDFs, policies, help articles, product specs - and cites exactly where each answer came from. That technique is called retrieval-augmented generation (RAG), and you will ship a production version without building a vector pipeline by hand.

Difficulty: Advanced. Expected build time: 2-4 hours across prompting, testing, and knowledge base prep.

Related: Add an AI assistant to your product covers the simpler single-doc case. Build an FMCG sales intelligence dashboard shows how to combine document retrieval with structured data queries.

Before you start

You need a Creatr account with Creatr AI Knowledge enabled. You also need a set of documents to ground the bot. Collect the real source material now - PDFs, spreadsheets, policy documents, help center exports, product manuals.

Two things to know before you gather files:

Quality in, quality out. The bot's answer ceiling is set by your documents. If your help docs are vague or outdated, the answers will be vague or outdated. No amount of prompt tuning compensates for weak source material. Start with whatever your best human support agent would actually read to answer a question.

Coverage gaps produce "I don't know." If a topic is not in the documents, a well-configured bot will say it does not know. That is correct behavior, not a failure. Make a list of the questions you expect the bot to answer and check that each topic is actually covered somewhere in your files before you upload.

Step 1 - Describe the assistant and its boundaries

Before uploading anything, write the core instruction. The single most important line is the boundary: answer only from the documents, and say so when the answer is not there.

Build a support chatbot that answers only from the documents I upload. If the answer is not in the documents, it should say it does not know rather than guess. Every answer should cite the document and section it came from. The bot should not offer opinions, recommendations, or information from outside the knowledge base.

This boundary is what separates a useful knowledge-base assistant from a generic chatbot. Without it, the model will fill gaps with its general training data and the answers will look confident and wrong.

If the assistant has a persona - a name, a tone, a specific product context - add that here too:

Name the assistant "Aria." It should respond in a professional but friendly tone. It represents [your company name] and only has knowledge of [your company name]'s products and policies.

Creatr wires this system instruction directly into every session so users cannot override it by asking the bot to ignore its rules.

Step 2 - Upload your knowledge base

Creatr AI Knowledge ingests your files into a scoped knowledge base attached to this assistant. Upload your files through the Creatr interface.

Practical advice on what to upload:

• PDFs and Word docs: product manuals, policy documents, onboarding guides, legal agreements.
• Spreadsheets: FAQs exported as CSV, product catalogs, pricing tables.
• Help center exports: if your team uses Notion, Confluence, or a help desk, export the relevant sections and upload them.

You do not need to pre-process the files. Creatr handles the pipeline internally. The next step explains what that pipeline does - understanding it will help you debug answers that go wrong.

Step 3 - Understand how retrieval actually works

Most builders skip this step and then cannot explain why the bot gives a bad answer. This explanation is plain English, not theory.

Chunking. Your documents are too large to hand to a language model in full. A 40-page PDF might contain 20,000 words; most models can process a few thousand at a time as useful context. So the pipeline splits each document into overlapping chunks - typically a few hundred words each, with some overlap so a sentence near the end of one chunk is also near the start of the next. Overlap prevents answers from falling through the cracks at chunk boundaries.

Embeddings. Each chunk is converted into a vector - a list of numbers that represents its meaning in high-dimensional space. Chunks about similar topics end up near each other in that space, even if they use different words. This is how a question about "refund policy" can retrieve a chunk that says "returns and reimbursements" without exact keyword overlap.

Hybrid retrieval. When a user asks a question, the system runs two searches in parallel. Semantic search finds chunks whose meaning is close to the question. Keyword search finds chunks that share exact terms with the question. The results are merged and scored. Hybrid retrieval outperforms either method alone, especially for queries that include specific names, product codes, or technical terms that semantic search treats as generic.

Reranking. The top results from retrieval are passed through a second model that re-scores them for relevance. Reranking catches cases where a chunk scored well in the vector search but is not actually useful for answering the specific question. Only the top-ranked chunks after reranking get passed to the language model as context.

Why this matters for debugging: if the bot gives a wrong answer or says it does not know when the answer is in your docs, the problem is almost always in this pipeline - a chunk that got cut at the wrong place, a topic that uses different terminology in the docs than users use in questions, or a document that was too densely formatted to parse cleanly. The fix is always in the source material, not in the prompt.

Step 4 - Test with real questions

Do not test with hypothetical questions. Pull the actual questions your users ask - from support tickets, search logs, sales call notes, or a list from your team.

Run each question through the bot and check three things:

1. Covered questions. Does the answer match the source document? Does it include a citation? Is the citation accurate - does clicking it take you to the right section?

2. Edge questions. Questions the docs cover partially. Does the bot give a partial answer and cite what it found, or does it fill the gap with a guess? It should do the former.

3. Out-of-scope questions. Questions the docs do not cover at all. Does the bot say it does not know, or does it invent an answer? If it invents, your boundary instruction from Step 1 needs to be stronger:

If the answer is not explicitly stated in the uploaded documents, respond with: "I don't have that information in the knowledge base. Please contact support at [email]." Do not infer, extrapolate, or use general knowledge.

Run at least 20-30 real questions before moving on. The test set you build here becomes your quality benchmark when you update the knowledge base later.

Step 5 - Fix bad answers at the source, not in the prompt

When the bot gives a wrong or unhelpful answer, the instinct is to rewrite the prompt. That rarely works. The problem is almost always the document.

Common causes of bad answers and the fix for each:

The answer is in the docs but the bot misses it. The chunk probably does not surface well in retrieval. Rewrite the relevant section in your source document to be more direct. Put the key fact in its own paragraph or short section. Dense, nested prose retrieves poorly. Short, clear sections retrieve well.

The answer is there but phrased differently than users ask. Add a short FAQ-style section to your source document that uses the exact language your users use. If users ask "how do I cancel" but your policy document says "termination of service," add a line that says "To cancel your subscription..." and mirrors the user's phrasing.

The bot hedges or gives an incomplete answer. The relevant chunk probably exists but sits at the edge of a chunk boundary, split away from the context that completes it. Check the surrounding paragraphs in the source document and make sure the complete answer is within a self-contained section - ideally under its own heading.

The bot gives an answer from general knowledge, not your docs. Your boundary instruction is not firm enough. Tighten it as shown in Step 4. Also check whether the topic exists in your uploaded documents at all - if it does not, the bot may be falling back on training data.

The pattern here is consistent: fix the source, not the model. Better documents produce better answers every time.

Step 6 - Turn on citations in the UI

Citations are what make the bot trustworthy. A user who can click a citation and verify the answer in the original document will trust the next answer more than one who gets a confident paragraph with no source. This is the operational difference between an assistant that creates confidence and one that creates doubt.

Show the source document name and section heading under each answer as a clickable citation. If an answer draws from multiple sections, show each source separately.

Creatr AI Chat renders citations as linked footnotes by default. You can adjust the display in your prompt - footnote style, inline parenthetical, or a "Sources" block below the answer.

If your documents are not publicly accessible, make sure the cited source shows enough context (document name, section title, page number if relevant) that a user can locate it in whatever system holds the original. A citation that says "Support Guide, Section 3.2" is more useful than one that says "Document 4."

Step 7 - Add access control if the docs are sensitive

If different users should only see answers drawn from documents their role permits, add scoping at the data layer now, before you ship.

RAG with data-layer access control means a query from a user in one role cannot retrieve chunks from documents that role is not allowed to see - even if the question would otherwise match those chunks. This is not prompt-level filtering; it is enforced at retrieval time.

Restrict each user so the chatbot only retrieves answers from documents their team has access to. Sales users should only see sales materials. Support users should only see support documentation. Admin users should see everything.

Describe your user roles and document categories in the prompt. Creatr maps the access rules to the knowledge base scopes at ingestion time.

If your documents are not sensitive and all users should see the same knowledge base, skip this step.

Step 8 - Keep the knowledge base up to date

A knowledge base that goes stale is as bad as a wrong answer. Users will start noticing that the bot cites outdated policies or does not know about recent product changes. The fix is a maintenance process, not a one-time upload.

Decide now how often your source documents change and set a matching update schedule:

• High churn (weekly product updates, frequent policy changes): assign someone to review the knowledge base weekly. When a document changes, re-upload the updated version and remove the old one.
• Low churn (stable documentation that changes quarterly): a monthly review pass is enough.

Add a prompt instruction that sets user expectations about currency:

If a user asks about something that may have changed recently, note that the knowledge base is updated on [your schedule] and suggest they verify with support for time-sensitive information.

When you re-upload a document, test the same questions from your Step 4 test set. Retrieval on the new version should produce the same or better answers. If something breaks, compare the old and new document versions - the section structure probably changed in a way that affected chunking.

Step 9 - Measure answer quality over time

Once the bot is live, measure whether it is actually working. Gut feel is not enough. Two things to track:

Deflection rate. What percentage of questions the bot answers do not go on to generate a support ticket? A working knowledge-base assistant should handle 40-60% of common questions without escalation. If deflection is low, either the coverage is poor (more documents needed) or the boundary is too aggressive (the bot is refusing questions it could answer).

Escalation reason. When users do escalate to a human, collect the reason. If a pattern of "bot did not know X" emerges around a topic that should be covered, that topic is missing or poorly represented in your documents.

Add a simple feedback loop to the chat UI:

Add a thumbs up / thumbs down rating under each bot response. Log the question, the answer, and the rating so I can review low-rated responses weekly.

Review the low-rated responses every week for the first month. You will find the gaps in coverage quickly. After the first month, most knowledge bases stabilize and monthly reviews are enough.

Step 10 - Ship it where users are

Drop the assistant into your product using Creatr AI Chat - a full page, a side panel, or an inline widget. Deploy when your test set passes at the rate you need.

Embed the assistant as a collapsible side panel on every page of the app. On mobile, show it as a full-screen overlay triggered by a chat button in the bottom right corner.

It goes live answering from your knowledge base, with citations, declining what it does not know, and scoped to the documents each user is allowed to see.

Recap

You shipped a grounded chatbot: answers from your own documents, citations on every response, a hard boundary against guessing, per-user access control, a process for keeping the knowledge base current, and a measurement loop to catch quality regressions early - without building or maintaining a vector pipeline.

The hard part of a trustworthy AI assistant is not the model. It is the retrieval, the source documents, and the boundaries. All three came handled. Your job going forward is keeping the source material accurate.

For next steps: if you want to combine this document-grounded chatbot with live data queries - for example, a bot that answers from policy docs and also looks up a user's account status in real time - see Add an AI assistant to your product.