Metrics with no model: exact match and string presence
Most RAGAS metrics ask a model to judge. Two do not, and they are the quickest way to see what a metric is: something that takes a sample and returns a number.
from ragas import SingleTurnSample
from ragas.metrics.collections import ExactMatch, StringPresence
exact = ExactMatch()
present = StringPresence()
for answer in ["3 March", "It shipped on 3 March."]:
sample = SingleTurnSample(user_input="When did A17 ship?", response=answer, reference="3 March")
print(answer)
print(" exact match ", exact.score(response=sample.response, reference=sample.reference).value)
print(" string present", present.score(response=sample.response, reference=sample.reference).value)Exact match compares the two strings character by character. The full sentence scores 0, even though it is the better answer, which is the reason judged metrics exist.
String presence asks whether the reference appears inside the answer. Both answers contain 3 March, so both score 1. It is the check you want for a required disclaimer, an order id, or a phone number that must appear.
Every metric in RAGAS returns a result object, not a bare number. result.value is the score; judged metrics also fill in result.reason, which lesson 5 uses.
- Score an answer of
"3 march", in lower case, with both metrics. - Swap the response and the reference around in
string presenceand explain the change.
You understood something today that you didn't yesterday.