Retrieval reduces hallucination. It does not remove it.
Retrieval-augmented generation (RAG) is the standard defense against hallucination: the model answers from retrieved documents rather than memory. But the evidence says that is not enough. In Stanford’s benchmark of commercial legal research tools, Lexis+ AI and Ask Practical Law AI “produced incorrect information more than 17% of the time,” and Westlaw’s AI-Assisted Research “hallucinated more than 34% of the time.” The authors concluded that “even RAG systems are not hallucination-free” (Stanford HAI).
NIST names the risk “confabulation”: “a phenomenon in which GAI systems generate and confidently present erroneous or false content” (NIST AI 600-1).
Long documents add another failure mode. Research on long contexts found performance “significantly degrades when models must access relevant information in the middle of long contexts” (Liu et al., TACL 2024). Compliance documents, such as 80-page SOC 2 reports or full design files, are exactly that kind of input.
The metrics that matter
| Metric | What it measures | Definition source |
|---|---|---|
| Faithfulness / groundedness | Whether “the claims that are made in the answer can be inferred from the context” | RAGAS paper; TruLens |
| Answer relevance | Whether “the generated answer should address the actual question that was provided” | RAGAS paper |
| Context relevance | Whether retrieved context is “focused, containing as little irrelevant information as possible” | RAGAS paper |
| Context precision | ”the retriever’s ability to rank relevant chunks higher than irrelevant ones” | Ragas docs |
| Context recall | ”how many of the relevant documents (or pieces of information) were successfully retrieved” | Ragas docs |
For compliance work, two of these matter most. Context recall catches the case where the evidence exists but was never retrieved, so the tool wrongly reports a gap. Groundedness catches the case where the tool cites a source that does not support its claim, which Stanford’s researchers described as a tool that “cites a source which does not in fact support its claims.”
Six controls that prevent hallucinated findings
- Passage-level citations. Every finding points to the exact page and passage. No citation, no finding.
- Claim-level grounding checks. Break each finding into claims and verify each is supported by the cited passage, the groundedness test above.
- Requirement-by-requirement assessment. Assess one clause at a time against focused context, rather than asking one question over a 200-page bundle, to limit long-context degradation.
- An honest “not found.” The system must be allowed, and expected, to say no evidence exists rather than stretch a weak match.
- Human review of every finding. Reviewers accept, edit or reject with a logged reason. NIST warns about “automation bias, or excessive deference to automated systems” (NIST AI 600-1), so review must be real, not a rubber stamp.
- Ongoing measurement. Track groundedness, context recall and reviewer override rates on a fixed test set, and re-run when models or prompts change.
An evaluation plan you can run in a pilot
| Step | What to do | Pass signal |
|---|---|---|
| Build a gold set | 50–100 requirement-document pairs with known answers, including some where evidence is absent | Covers met, partial, not met and not applicable |
| Measure citation accuracy | Check whether each cited passage supports the finding | High share of supported citations; every miss explained |
| Measure recall | Count known evidence the tool failed to find | Few false “gaps” |
| Test “not found” | Include requirements the documents do not address | Tool says not found rather than inventing evidence |
| Stress long documents | Place key evidence mid-document in long files | No drop in recall for mid-document evidence |
| Track overrides | Log reviewer edits and rejections | Stable or falling override rate |
Set the thresholds before the pilot and document them, in line with FDA’s risk-based approach to validating AI tools used in quality systems (FDA CSA guidance).
Evidence first, by design
AskDona is built around these controls: it assesses documents requirement by requirement, cites the passage behind every finding and leaves the decision with your reviewers.
Run a sample document assessment with documents your team already knows, and check every citation yourself.
FAQ
Does RAG eliminate hallucinations? No. Stanford found commercial RAG-based legal tools still gave incorrect information in more than 17% of benchmark queries (Stanford HAI).
What is groundedness? Whether every claim in an AI answer is supported by the retrieved source text (TruLens).
Why do long documents cause problems? Models perform worse on information in the middle of long contexts (Liu et al.), so focused, requirement-level retrieval helps.
What is the most important safeguard? Passage-level citations combined with real human review.
References
All pages opened and checked by October 9, 2026.
| Source | Type | Date | Used for |
|---|---|---|---|
| Stanford HAI: AI on Trial | Academic | May 23, 2024 | RAG hallucination rates |
| NIST AI 600-1: Generative AI Profile | Standard | Jul 2024 | Confabulation, automation bias |
| Liu et al.: Lost in the Middle (TACL 2024) | Peer-reviewed | 2024 | Long-context degradation |
| Es et al.: RAGAS (arXiv) | Preprint | Sep 26, 2023 | Faithfulness, relevance |
| Ragas docs: Context precision | Tool documentation | Current | Context precision |
| Ragas docs: Context recall | Tool documentation | Current | Context recall |
| TruLens: RAG triad | Tool documentation | Current | Groundedness |
| FDA: CSA guidance | Regulator guidance | Feb 3, 2026 | Risk-based AI tool validation |