A JavaScript assertion

A JavaScript assertion is the same idea as the Python one in Node: type: javascript with value: file://assert.js, exporting a function that returns a verdict.

Last updated: 15 Oct, 2026 · Promptfoo 0.124.1

Promptfoo runs on Node, so a check written in JavaScript needs nothing extra installed. The file exports one function, promptfoo calls it with the answer and a context, and it returns either a boolean or a {pass, score, reason} object, the same two shapes the Python check uses.

The exported function

The entry point is the module's export itself, called as (output, context). This check pulls month names out of the answer and reports which it found. The syntax is plain Node, so string and array methods are all there.

// a JavaScript assertion returns {pass, score, reason} (or a plain true/false)
const MONTHS = ["January", "February", "March", "April", "May", "June"];

module.exports = (output, context) => {
  const named = MONTHS.filter((m) => output.includes(m));
  return {
    pass: named.length > 0,
    score: named.length ? 1 : 0,
    reason: named.length ? `names ${named.join(", ")}` : "no month named",
  };
};

The config points a javascript assertion at the file, across two questions: one the bot answers with a date, one it misses.

providers:
  - file://support_bot.py
prompts:
  - "{{question}}"
tests:
  - vars: { question: "Where is order A17?" }
    assert:
      - type: javascript
        value: file://assert.js
  - vars: { question: "Is the store open on Sunday?" }
    assert:
      - type: javascript
        value: file://assert.js

# read out.json: each answer, its checks, and any metric scores
import json

for r in json.load(open("out.json"))["results"]["results"]:
    print(r["response"]["output"], "->", "PASS" if r["success"] else "FAIL")

    for c in r["gradingResult"]["componentResults"]:
        print("  ", "pass" if c["pass"] else "FAIL", c["assertion"]["type"], "-", c["reason"])

    if r["namedScores"]:
        scores = {k: round(v, 3) for k, v in r["namedScores"].items()}
        print("   metrics", scores, "| test score", round(r["score"], 3))

# the shop support bot: find the answer in the handbook, or say it is missing
HANDBOOK = {
    "a17": "Order A17 shipped on 3 March by courier.",
    "refund": "Refunds take five working days.",
}

def call_api(prompt, options, context):
    asked = prompt.lower()                      # the rendered prompt is the question
    for key, line in HANDBOOK.items():
        if key in asked:
            return {"output": line}
    return {"output": "I could not find that in the handbook."}

promptfoo eval -c jsassert.yaml -o out.json --no-table >/dev/null 2>&1
python3 show.py

Order A17 shipped on 3 March by courier. -> PASS
   pass javascript - names March
I could not find that in the handbook. -> FAIL
   FAIL javascript - no month named

The JavaScript entry point

Python or JavaScript

Where a JavaScript assertion fits

• A Node codebase, where the check sits next to the app it tests in the same language.
• An npm library you want to call from inside the check, such as a date or schema helper.
• A quick inline rule, where the logic is a line or two of JavaScript on the answer.

Related

• Previous: A Python assertion: a check that is a function
• Next: Transforms: reshaping output before a check
• Reference: Promptfoo JavaScript assertions