RAGASragas 0.4.3 · Python 3.10+
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
28 small wins to finish your pathNext lesson →

Custom metrics

A custom metric in RAGAS is a plain Python function turned into a metric by the discrete_metric or numeric_metric decorator, so it scores alongside the built-in ones with no judge.

Last updated: 29 Sep, 2026 · RAGAS 0.4.3

The five RAG metrics cover what every RAG app needs. The rules only your product has, such as "every price answer shows a dollar sign", are usually one line of Python. A decorator makes that line a metric.

The discrete_metric and numeric_metric API

python
from ragas.metrics import discrete_metric, numeric_metric

@discrete_metric(name="...", allowed_values=["pass", "fail"])  # returns one of a fixed list
def my_check(response: str) -> str: ...

@numeric_metric(name="...", allowed_values=(0, 1))             # returns a number in a range
def my_score(response: str) -> float: ...

A price check for TechNest

TechNest's price answers must show a dollar amount. The function returns pass or fail.

python
@discrete_metric(name="shows_price", allowed_values=["pass", "fail"])
def shows_price(response: str) -> str:
    return "pass" if re.search(r"\$\d", response) else "fail"

An answer-length score

The app's prompt asks for two or three sentences. Forty words counts as full length here; shorter answers score a fraction.

python
@numeric_metric(name="answer_length", allowed_values=(0, 1))
def answer_length(response: str) -> float:
    return min(len(response.split()) / 40, 1.0)

Two custom metrics on two answers

Example
import re

from ragas.metrics import discrete_metric, numeric_metric


@discrete_metric(name="shows_price", allowed_values=["pass", "fail"])
def shows_price(response: str) -> str:
    return "pass" if re.search(r"\$\d", response) else "fail"


@numeric_metric(name="answer_length", allowed_values=(0, 1))
def answer_length(response: str) -> float:
    return min(len(response.split()) / 40, 1.0)


for text in ["The PixelPhone 15 is priced at $899.", "It costs eight hundred ninety nine dollars."]:
    print(shows_price.score(response=text).value, answer_length.score(response=text).value, "|", text)

What the decorated functions returned

  • shows_price passes the first answer, which has $899, and fails the second, which spells the price out.
  • answer_length gives each answer its word count over 40, a fraction because both are short.
  • .score(...).value reads the result the same way as for every built-in metric, so these can sit in the same experiment.

discrete_metric vs numeric_metric

discrete_metricnumeric_metric
ReturnsOne value from allowed_valuesA number inside allowed_values
Example valuespass or fail, safe or unsafe0 to 1
Good forRules with a yes or no answerMeasures such as length or count

Checks worth writing as code

  • A required disclaimer, a currency symbol, an order id format, or valid JSON.
  • A maximum length, a banned word, or a link that must be present.
Watch out. A function metric only sees the arguments you name. shows_price reads response; if you want it to compare with the golden, add reference: str to its signature and pass it to score.
Try it yourself
  • Write a discrete_metric that fails any answer longer than 60 words.
  • Give shows_price a third allowed value, "wrong", returned when the answer shows a price that is not $899.
PreviousFactual correctness

You understood something today that you didn't yesterday.