Agent Governance ToolkitAgent Governance Toolkit 4.1.0 · Python 3.10+
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
47 small wins to finish your pathNext lesson →

Testing a policy from the command line

agt test replays a set of fixtures against a policy and reports any decision that does not match what the fixture expected, returning a non-zero exit code on a mismatch.

Last updated: 14 Oct, 2026 · Agent Governance Toolkit 4.1.0

The lint-policy lesson checked a policy's shape. Shape is not behaviour. agt test runs the policy against named inputs and checks the verdicts, which is what you gate a CI pipeline on.

A policy and a set of fixtures

The policy is the shop policy. A fixture names an input and the verdict it expects: a status check should be allowed, an account deletion should be blocked. Save the policy as orders-policy.yaml and the fixtures as orders-fixtures.json.

yaml
version: "1.0"
name: shop-policy
rules:
  - name: allow-status
    condition: {field: action, operator: eq, value: order_status}
    action: allow
    priority: 10
  - name: block-delete
    condition: {field: action, operator: eq, value: delete_account}
    action: deny
    priority: 100
defaults:
  action: deny
json
[
  {"name": "status is allowed", "input": {"action": "order_status"}, "expected_allowed": true},
  {"name": "delete is blocked", "input": {"action": "delete_account"}, "expected_allowed": false}
]

agt test takes the policy and the fixtures, and replays each one.

ExampleBoth fixtures get the verdict they expect
agt test orders-policy.yaml orders-fixtures.json

A fixture that expects the wrong verdict

This fixture expects an account deletion to be allowed, which the policy denies. The run reports the mismatch, names the rule that decided it, and exits with code 1 so a pipeline fails. Save it as bad.json.

json
[
  {"name": "expect delete allowed", "input": {"action": "delete_account"}, "expected_allowed": true}
]
ExampleA mismatch names the rule and fails the run
agt test orders-policy.yaml bad.json

What the run reports

  • One line per fixture, ok when the verdict matched and FAIL when it did not.
  • The wanted and the actual verdict on a failure, with the rule that produced it, so you can see which rule to fix.
  • A non-zero exit code on any mismatch, which is what lets CI block the change.

The fixture fields

FieldMeans
nameA label for the fixture in the report
inputThe context the policy is evaluated against
expected_allowedThe verdict the fixture expects, true or false

Where you use agt test

  • A regression suite for a policy, so a change that flips a verdict is caught.
  • A CI gate on every policy edit, failing the build on a mismatch.
  • A worked specification, where the fixtures document what the policy is meant to do.
Watch out. agt test uses the real policy loader, which silently drops rules it does not understand. A fixture can pass because the rule meant to block it was dropped and the default allowed it. Keep a fixture that expects a block for every rule that is meant to block, so a dropped rule shows up as a mismatch.
Try it yourself
  • Add a fixture for a refund and give it the verdict your policy should return.
  • Change the policy to allow deletions and re-run the original fixtures to see them fail.
  • Put both fixture files in a directory and pass the directory instead of a single file.
PreviousLinting a policy before it ships

Slow is fine. Stopping is the only problem.