RAG evaluation: how do you test that citations hold up?

01Why RAG evaluation needs citation tests

Citations are how users decide to trust an answer without reading the source. That is their value, and it is also the risk. A user who sees a link next to a sentence assumes someone checked it.

Research on public generative search engines shows how often that assumption fails. An audit of four such systems by Liu, Zhang and Liang found that only about half of generated sentences were fully supported by their citations. It also found that roughly a quarter of citations did not support the sentence they were attached to. Those were well-funded consumer products. A RAG feature inside a vertical SaaS product, on modest models and messy customer documents, will not do better by default.

So "the answer has citations" is not a test. The test is whether each citation does its job.

02Separate retrieval from generation

Before you can test citations, you need a pipeline you can test in parts. Our preference, whenever a product depends on cited research or cited documents, is to keep retrieval as a plain, inspectable step. That step is a search over an index or a search API that returns passages and their sources. The language model comes after, as a separate step that writes from those passages.

Answer-style products that search, read and write in one call are convenient, but they hide the step you most need to check. If the system returns a finished answer with links, you cannot tell whether a bad citation came from poor retrieval, poor ranking or the model attaching the wrong source. With the steps separate, each one gets its own test:

• Retrieval. Did the right passages come back for this question, within the user's permissions?
• Ranking. Were the passages that answer the question ranked high enough to be used?
• Generation. Did the model use only those passages, and does each claim cite the passage it came from?
• Refusal. When no passage answers the question, did the system say so?

03Test claims, not answers

The core metric is support per claim. Frameworks such as Ragas define faithfulness this way: break the response into claims, check each claim against the retrieved context, and divide the number of supported claims by the total. A two-claim answer with one wrong date scores 0.5, which is exactly the signal you want.

For citations, go one step further than faithfulness to the whole context. Check each claim against the specific passage it cites. A claim can be true according to some retrieved passage and still cite the wrong one, which sends the user to a page that does not back it up.

A practical evaluation run looks like this:

1. 01

   Build a question set from real documents

   , including questions written by the people who will use the feature, not only by engineers.

2. 02

   Write expected answers and expected sources

   for each question, including questions whose correct answer is "the documents do not say".

3. 03

   Run the pipeline

   and store the retrieved passages, the answer and the citations for each question.

4. 04

   Split each answer into claims

   and pair each claim with its cited passage.

5. 05

   Judge support

   for each pair with a model-based judge, and have people check a sample of the judge's decisions so you know how far to trust it.

6. 06

   Score separately

   for retrieval hit rate, citation support, refusal accuracy and permission leaks, and track each over time.

04The cases that break cited answers

Average scores hide the failures users remember. These are the cases we test on purpose:

The last row deserves care. If permission filtering happens after ranking or after generation, a restricted passage can shape the answer even when its link is hidden. Filter by tenant, collection and clearance before ranking, apply the same scope to citations and counts, and fail closed when a user lookup fails.

05Make refusal a feature

The single most valuable behavior in a cited system is a clear "the documents do not answer this". It is also the hardest to keep, because models are trained to be helpful and a near-match always seems close enough.

Write refusal into the evaluation set from day one, at a meaningful share of questions. Score it as its own metric. When a change to prompts, chunking or models raises answer rate, check that refusal accuracy did not fall at the same time. Often it did, and the higher answer rate is the system guessing.

The economics of these choices, including when a larger self-hosted model is worth its GPU memory, are covered in how much production AI costs in SaaS.

06What we see in production

For a document-management software company serving government archives, we built a self-hosted, cited knowledge workspace over sensitive French and English records, and later an AI gateway that the client's own document-management system calls. Everything runs on self-hosted models; no document goes to an outside AI service. Answers must cite the passage they came from, decline when the documents do not cover a question, and never substitute a near-match entity.

The hard part was that citation quality depends on every earlier step lining up. Those steps are OCR, chunking of lists and tables in Word files, entity extraction, permission filtering, ranking and citation alignment, all on modest models sharing one GPU. Archival scans, near-identical names, office holders listed in bullets and aggregate questions all pushed the model toward confident wrong answers. We chose the OCR engine after side-by-side tests, filtered retrieval by tenant, collection and clearance before reranking, and made unknown or deactivated users fail closed.

The writing tools in the same workspace follow that discipline. An approved term registry, loaded as a custom dictionary in English and French, keeps reviewed names from being changed. Paraphrase pins names, numbers and attributions and checks them again afterward, and the same registry also protects terms in the workspace's answers.

The client tested rigorously with purpose-built documents and real users, and each test round became tracked findings that we fixed and re-verified. The client's own tests confirmed that the system stopped inventing names when documents lack the answer and declines instead of describing a similar entity. Some retrieval items, such as bulleted lists in Word files and full table rows, were still being worked on, which is normal: citation quality is something you keep testing, not something you finish.

Frequently asked questions