RAG Tutorial: How to Build a Retrieval-Augmented Generation App with Reliable Citations

Overview

Retrieval-augmented generation (RAG) combines information retrieval with an LLM. Instead of asking a model to answer from its general training alone, the application first searches a collection of documents, selects relevant passages, and includes them in the model request. The result can be more grounded in your own policies, manuals, knowledge base, or product documentation.

A reliable RAG system is more than a vector database connected to a chatbot. Its quality depends on the whole path from source documents to final answer: ingestion, parsing, chunking, embeddings, search, filtering, prompt assembly, citation handling, and evaluation. A weakness at any stage can produce missing context, irrelevant results, duplicate evidence, or confident answers that the source material does not support.

Use the checklist below when planning a new LLM application, changing a retrieval workflow, or reviewing an existing system. The examples assume a text-based knowledge base, but the same principles apply to support assistants, internal search tools, content operations, and other AI development projects.

Checklist by scenario

1. Before ingesting documents

• Define the answerable scope. Write down which questions the application should answer and which questions it should decline or route elsewhere. A narrow, explicit scope makes retrieval and evaluation easier.
• Identify authoritative sources. Prefer approved documents and record their title, owner, publication date, version, and access permissions. Do not treat every file in a shared folder as equally reliable.
• Preserve useful structure. Extract headings, lists, tables, page numbers, section names, and document identifiers where possible. Structure becomes valuable metadata for filtering and citations.
• Remove or label obsolete content. If old and current versions remain searchable, the model may receive conflicting passages. Either archive outdated material or attach clear version and status metadata.
• Plan for sensitive data. Decide what may be indexed, who may retrieve it, and how access controls will be enforced. Retrieval should not bypass the permissions applied to the original source.

2. When choosing a chunking strategy

• Start with semantic boundaries. Split by headings, sections, procedures, or paragraphs before relying on a fixed character or token size. A chunk should represent a coherent idea.
• Keep enough context to preserve meaning. Very small chunks can lose definitions and conditions; very large chunks can dilute search relevance and consume the model's context window.
• Use overlap deliberately. Overlap can preserve a sentence that crosses a boundary, but excessive overlap increases duplicate results and storage. Test it against representative questions rather than selecting it by habit.
• Keep citation fields with each chunk. Store a stable source ID, document title, section, URL or file reference, and location such as a page or heading.

3. When configuring embeddings and vector search

• Match the embedding model to the content. Consider language, terminology, document types, and query style. Technical acronyms and product names deserve explicit testing.
• Index the same representation you intend to search. If documents are heavily transformed during ingestion, verify that the stored text remains understandable when returned to the generation step.
• Begin with a modest retrieval count. Retrieve enough candidates to cover likely evidence, then inspect whether additional results improve recall or merely add noise. The best value depends on document length, question complexity, and model context limits.
• Combine semantic and lexical search when useful. Vector search can capture related meaning, while keyword or full-text search can help with exact product names, codes, and identifiers. A hybrid approach is worth testing for terminology-heavy collections.
• Apply metadata filters early. Filter by tenant, permission, product, language, document type, or version before or during retrieval when the system supports it. This reduces irrelevant evidence and supports data separation.

4. When assembling the answer prompt

• Separate instructions from evidence. Clearly label the user's question, retrieved context, and response requirements. This makes the prompt easier to inspect and reduces ambiguity.
• Tell the model how to handle uncertainty. Instruct it to say when the supplied context is insufficient rather than filling gaps with an unsupported answer.
• Require source references. Ask the model to attach a citation to claims that rely on retrieved material. Use stable source labels such as [1], [2], and map them to document details in the interface.
• Limit citation scope. A citation should support the specific claim beside it, not simply appear at the end of a long response. If a passage supports only one condition or exception, the wording should reflect that.
• Make the output format predictable. Define headings, bullet rules, refusal wording, and citation syntax when downstream systems need to parse the response.

5. When evaluating the application

• Create a representative test set. Include straightforward questions, ambiguous questions, questions requiring several documents, questions about exceptions, and questions with no answer in the collection.
• Evaluate retrieval separately from generation. Check whether the correct source appears in the retrieved set before judging the wording of the final answer. A generation prompt cannot reliably compensate for absent evidence.
• Review citation correctness. Confirm that each cited passage supports the associated claim and that the source shown to the user is the source actually retrieved.
• Record failure categories. Label errors as missing context, irrelevant context, stale source, duplicate result, incorrect synthesis, unsupported claim, or formatting failure. Categories point to different fixes.

What to double-check

Chunk boundaries: Ask whether a reviewer could understand each retrieved passage without opening the original document. If not, preserve more surrounding context or include the parent heading.

Retrieval settings: Compare several retrieval counts and, where available, similarity thresholds or reranking settings. A high result count is not automatically better; it can bury the strongest evidence in noise.

Metadata filters: Test permissions and version filters with documents that share similar wording. Ensure a user cannot receive a result from another tenant, department, region, or restricted collection.

Citations: Check that citations remain correct after documents are reprocessed. Stable IDs are usually safer than relying only on changing chunk positions.

Stale documents: Test a question where an old policy conflicts with a current one. The system should prefer the intended source according to explicit status, date, or version rules.

Prompt injection in retrieved text: Treat document content as data, not as instructions. A retrieved passage may contain text that attempts to alter the application's behaviour. Keep system and application instructions separate, and test hostile or unexpected document content.

Operational visibility: Log the query, filters, retrieved source IDs, scores or rankings where available, prompt version, and final response subject to your privacy requirements. Without this trail, diagnosing a bad answer becomes guesswork.

Common mistakes

• Indexing everything without ownership rules. More documents create a larger search space, not necessarily a better knowledge base.
• Using fixed-size chunks for every document. A legal clause, product table, and troubleshooting procedure often need different structural treatment.
• Increasing top-k to hide poor retrieval. If results are irrelevant, inspect parsing, chunking, query formulation, and metadata before simply returning more passages.
• Confusing a fluent answer with a supported answer. Review evidence and citations, not just grammar and usefulness.
• Allowing the model to invent citations. Generate citation labels from retrieved records where possible, and validate that every label maps to a real source.
• Skipping no-answer tests. A dependable application needs a clear response when the knowledge base does not contain the requested information.
• Changing several variables at once. When testing a RAG system, change one major factor at a time—such as chunk size, retrieval count, or prompt wording—so the result can be interpreted.

For the wider production context, pair this RAG checklist with a production checklist for deploying an LLM app and a structured prompt engineering evaluation framework.

When to revisit

Revisit the RAG workflow whenever its inputs or operating conditions change. Schedule a review before seasonal planning cycles if documents, policies, products, or support volumes are likely to change. Also review it after a major document migration, embedding or model change, vector-store change, permission update, or prompt revision.

At each review, run the same labelled test set and compare retrieval coverage, citation correctness, answer completeness, refusal behaviour, latency, and cost against the previous version. Add real failures to the test set, but remove confidential details or anonymise them where appropriate.

A practical update sequence is:

1. Confirm which documents and versions should be searchable.
2. Reprocess a controlled sample and inspect chunk boundaries and metadata.
3. Run retrieval-only tests before assessing generated answers.
4. Check citations and no-answer behaviour with known edge cases.
5. Review access controls, logs, and retention settings.
6. Record the new configuration, test results, and unresolved limitations.

Use this checklist as a release gate for meaningful changes rather than as a one-time setup task. RAG quality is maintained by keeping sources current, retrieval behaviour observable, prompts controlled, and citations tied to evidence that a user can inspect.