What Is RAG? Check Retrieval before Trusting AI Answers

Why does AI still get conditions wrong after we attach a manual? Having documents in the system does not mean every passage reaches the model. RAG retrieves relevant portions, but failures can occur during both retrieval and answering.

Documents pass from a library through selection into an answer, with gaps representing potentially missing context

RAG retrieves evidence to support answer generation

Retrieval-Augmented Generation or RAG uses retrieved evidence as context for answer generation alongside knowledge in the model’s weights. The RAG of Lewis et al. is an important original source for this concept. Current production systems may use structures and retrieval methods differing from the original research.

Imagine an assistant looking up a service manual, selecting relevant pages, and using them to answer. Choosing the wrong version or half a paragraph leaves gaps that good writing cannot repair.

RAG is not equivalent to training a new model. And it is not a collective name for all types of file attachments. One system may send the entire document into the context without searching. Another system may search the text. Some systems may also read images of document pages. Explaining the system requires looking at the actual data path. Don't judge by the words "Talk to PDF" on the ad page.

How RAG retrieves text from files and passes it to the model

The question enters the Retrieve stage to select documents. Then the document and questions are entered into Prompt and LLM before creating the Answer.
LangChain diagram separating document selection in Retrieve from Answer generation through Prompt and LLM. Missing evidence may result from failure before answering. This is neither an accuracy test nor a diagram from the original Lewis research. Source: LangChain Docs Used according to MIT Resized proportionally and converted to WebP without text changes
  1. Prepare content: Extract text or data from documents with file name, version, date, and access permissions
  2. Split: Split into searchable chunks while retaining necessary headings and conditions
  3. Build an index: Create a structure for searching with exact words and meanings through embedding or a combination of methods.
  4. Search when you have questions: Select relevant chunks, potentially filtering permissions and document versions and reranking results
  5. Generate the answer: Send selected evidence to the model with instructions on scope and citations
  6. Verify the answer: Check that claims are supported by evidence the system actually used.

Embeddings represent data numerically to help find relationships; they do not certify document truth. Exact search may suit product codes, while semantic similarity helps differently worded questions. Neither is always correct.

Anthropic's Contextual Retrieval article explains context lost through chunking and combining semantic with literal search. KrobClick uses these ideas to design system-checking questions, without treating developer benchmark figures as guarantees for Thai results.

Hypothetical example: the condition is in a different section from the answer

Suppose a store manual has two passages: “Products may be returned within 7 days of receipt” and a later note, “Excluding items custom-made to personal measurements.” A customer asks, “Can a made-to-measure shirt be returned?” These statements and time periods are hypothetical examples explaining the system, not KrobClick store terms or an explanation of legal rights.

What to observeLayers to checkWhat to fix or test
Retrieves only the sentence saying returns are accepted within 7 daysData preparation and retrievalWere notes imported, and can they be associated with the same topic?
Retrieves all conditions and exceptions but claims returns are allowed in every casegenerating answersInstructions and the ability to interpret exceptions from evidence
Answers from an old manual with different conditionsVersion and metadataFilter the effective version and remove old copies from the response set.
Answers correctly but accesses a document the questioner is not authorized to viewPermission controlsEnforce access permissions before including data in answers

Larger models may interpret better but cannot fix never-imported documents or incorrect search permissions. Inspect both answers and retrieved text to target the failing layer.

Thai questions must be tested with words that people actually use.

The manual may say: “Cancel the order” but the user asks “I changed my mind and don't want the item, what should I do?” There should be both formal and colloquial language questions in the test kit. The other group is a code, number, and model name that differs only by one character. This should check for direct matches instead of assuming that the meanings are close enough.

Prepare answerable questions, questions requiring exceptions, unanswerable questions, and conflicting documents. Specify expected evidence before testing so convincing answers do not shift your criteria.

For a small test set, report the number of items and specific errors. Do not describe a pass rate from handpicked examples as overall task accuracy. Retest when changing the model, document segmentation, or data collection, as these conditions can change behavior.

Evaluate searches and answers separately.

A simple log is “Question / Expected documents / Retrieved documents / Answer / Errors / Reviewer.” If retrieval results are inaccessible, record that limitation rather than diagnosing from the final answer alone.

Use RAG for data needs, not because it is fashionable

Frequently changing documents, citation needs, or differing permissions can justify retrieval augmentation. A single short document may need only direct context. More complexity adds index, data-version, permission, and evaluation maintenance.

Before choosing technology, identify where the correct information is and how to verify that the system used it. This helps determine whether retrieval, document organization, or answer checking needs work. A longer prompt alone may not address the cause.

Read moreAI Context, Memory, and Chat History: What Is the Difference?Stored history, saved memory, and the context a model sees for the current response need separate checks.

Sources

Sources reviewed October 14, 2026, Thailand time. Thai examples and prompts are KrobClick proposals, not product tests. Features and settings vary by version/account. Check originals and outputs before real use.