Ragas vs TruLens: did retrieval miss a clause, or did the answer cite the wrong evidence?

Ragas is better suited to checking required fact coverage with reference answers; TruLens is better suited to tracing evidence loss and misattribution across a request. Separate recall, faithfulness, and citation attribution, and preserve the context the model actually saw.

Source: docs.ragas.io
November 15, 2026
6 min

After a customer-service knowledge base reduces retrieval from five passages to three, refund answers become shorter but omit eligibility conditions. Did retrieval miss the clause, or did the model receive it and then write or cite it incorrectly? Looking only at an overall score for the final answer often assigns the repair to the wrong person.

Evaluate Ragas first when you have reference answers and need to compare retrieval configurations. Evaluate TruLens first when you need to trace where evidence disappears within a request or verify citation targets. This recommendation is based on official documentation, not a hands-on ranking. Both now go beyond RAG evaluation alone: the Ragas documentation covers agent and workflow evaluation, while TruLens positions itself around agent evaluation and tracing.

First identify whether evidence is missing or being misused

Ragas ContextRecall takes the question, user_input, the retrieved passages, retrieved_contexts, and a reference answer, reference. A judge model determines whether the passages cover the claims in that reference. The score is the proportion of claims covered; higher is better. It suits the question “Did we retrieve all the necessary conditions?” The cost is maintaining trustworthy reference answers. If a reference omits an exception, the recall score cannot see that gap either.

ContextPrecision instead examines whether useful passages rank near the top. The current collections API likewise supports scoring with a question, passages, and reference answer. Without a reference answer, ContextUtilization uses the generated response to judge passage usefulness. It cannot be treated as coverage of a known correct answer. Relevance, ranking, and recall are different questions: all three passages can be relevant while still missing the refund exception.

The TruLens RAG triad separately examines question–passage relevance, consistency between the answer and evidence, and whether the answer addresses the question. All three work without reference answers, making them useful for initially screening unlabeled requests. But high relevance cannot prove that an unretrieved clause does not exist. TruLens also has a retrieval evaluation workflow that uses golden passages to calculate recall and NDCG. The two products should not be reduced to an absolute division between “with references” and “without references.”

A supported answer can still cite the wrong passage

Ragas Faithfulness takes the question, response, and retrieved passages, without requiring a reference answer. It decomposes the response into claims and calculates the proportion supported by the passages, on a 0–1 scale where higher is better. TruLens groundedness likewise checks whether the claims in an answer have support in the supplied context. Neither type of metric alone proves that the answer is complete or that the knowledge base itself is correct.

A subtler error occurs when the refund deadline really does appear in passage two, but the answer cites passage one. The evidence set supports the sentence, so a consistency score may still look good. TruLens citation templates explicitly separate two checks. CitationAttribution checks explicit [N] markers against numbered passages; misattribution fails the check, but claims without markers are not penalized. CitationAccuracy supports a wider range of citation formats, grades on a 0–3 scale, and counts missing required citations as a problem. Do not directly subtract its raw grades from 0–1 scores.

Therefore, when showing source links to customer-service staff, selecting a “faithfulness” metric alone is insufficient. Teams using Ragas need a separate citation-target check. Teams using TruLens must still choose the appropriate template and preserve the mapping between numbers and original passages. Neither product can decide which sentences your business requires to carry citations.

Separate responsibilities with a reproducible record

The following is a proposed evaluation design, not an executed test. Prepare 30 human-validated refund questions covering standard refunds, exceptions, and cases where the knowledge base has no answer. For each record, save the question ID, source-document version, reference answer, and required evidence passages. For each run, separately save the ordered retrieval list, context actually sent to the model, full response, citation mapping, and configuration version.

For example, ask: “I activated an annual plan 10 days ago and have already used the service. Can I still receive a full refund?” Define synthetic test clauses as follows: [1] A refund can be requested within 14 days of activation. [2] A full refund is not available if the service has been used. The reference answer must include both clauses and their relationship. This is a synthetic business example, not any company's refund policy.

Freeze the knowledge-base snapshot, generation model, and prompt, changing only retrieval top-k from 5 to 3. First give both sets of actual context to Ragas ContextRecall, then check Faithfulness. In TruLens, evaluate relevance per passage and groundedness against the full context and response, then separately check citations. Its Selector documentation explains that checking passages individually and collecting context are different choices. If retrieval is followed by reranking, filtering, or truncation, record those steps. Do not treat candidate passages the model never saw as its evidence.

This record supports at least four routes for repair. If clause two never enters the candidate list, inspect retrieval. If it enters the candidates but not the model context, inspect context assembly. If the model sees both clauses yet promises a full refund, inspect generation. If the answer is correct but cites the wrong number, inspect attribution. If the source in the knowledge base is outdated, send it to the content owner; changing judges will not fix it.

Fix the judge and criteria before comparing scores

Both products' LLM metrics depend on the judge. Ragas metric examples explicitly supply a judge model, and TruLens provider configuration also allows model selection. For comparison, fix the same available judge version, temperature, and input text, and record each product's prompts, metric implementation, and aggregation method. Even with the same model, 0.8 under different rubrics is not the same measurement.

First ask business reviewers to label the examples as “missing evidence,” “contradicting evidence,” “misattribution,” or “should abstain,” then check whether automated evaluation catches those errors. Repeat scoring for borderline cases and preserve the reasons. Count misses by category rather than letting one average conceal high-risk promises. Cases with no answer should be checked for appropriate abstention, not forced into recall scoring with an invented reference answer.

If the bottleneck is proving that new retrieval has not omitted necessary facts, Ragas reference-answer metrics offer a more direct route. If the bottleneck is disputes over responsibility across reranking, assembly, and generation, TruLens step selection and citation checks better fit the diagnosis. First capture the actual context and annotate the evidence, then choose the tool. Otherwise, both can produce attractive scores without telling you which step needs repair.

Content note:This article is based on publicly available information and is intended as a reference for evaluating tools. Features, pricing, and related policies may change at any time; please refer to the latest official information.

Original source:docs.ragas.io