RAGASragas 0.4.3 · Python 3.9+
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
20 small wins to finish your pathNext lesson

Metrics with no model: exact match and string presence

Most RAGAS metrics ask a model to judge. Two do not, and they are the quickest way to see what a metric is: something that takes a sample and returns a number.

Example
from ragas import SingleTurnSample
from ragas.metrics.collections import ExactMatch, StringPresence

exact = ExactMatch()
present = StringPresence()

for answer in ["3 March", "It shipped on 3 March."]:
    sample = SingleTurnSample(user_input="When did A17 ship?", response=answer, reference="3 March")
    print(answer)
    print("   exact match   ", exact.score(response=sample.response, reference=sample.reference).value)
    print("   string present", present.score(response=sample.response, reference=sample.reference).value)

Exact match compares the two strings character by character. The full sentence scores 0, even though it is the better answer, which is the reason judged metrics exist.

String presence asks whether the reference appears inside the answer. Both answers contain 3 March, so both score 1. It is the check you want for a required disclaimer, an order id, or a phone number that must appear.

Every metric in RAGAS returns a result object, not a bare number. result.value is the score; judged metrics also fill in result.reason, which lesson 5 uses.

Where these fit
Deterministic metrics cost nothing and never disagree with themselves. Use them for anything with one right answer, and save the judged metrics for the parts that need reading.
Try it yourself
  • Score an answer of "3 march", in lower case, with both metrics.
  • Swap the response and the reference around in string presence and explain the change.

You understood something today that you didn't yesterday.