Agent Governance ToolkitAgent Governance Toolkit 4.1.0 · Python 3.10+
Dashboard
0%

Testing the policy in CI

Testing the policy in CI means running a fixed set of fixtures against policy.yaml with agt test on every change, so a rule that stops blocking an attack fails the build.

Last updated: 14 Oct, 2026 · Agent Governance Toolkit 4.1.0

The governed agent is only as good as its policy, and a policy is a file that people edit. The command-line agt test turns the shop's rules into a test you can gate a pull request on.

The policy and its fixtures

The policy is the shop policy from the project. The fixtures cover every rule: the two allowed tools, the blocked deletion, and an unknown tool that should fall through to the deny default. Save them as policy.yaml and fixtures.json.

yaml
version: "1.0"
name: shop-policy
rules:
  - name: allow-status
    condition: {field: action, operator: eq, value: order_status}
    action: allow
    priority: 10
  - name: allow-refund
    condition: {field: action, operator: eq, value: refund}
    action: allow
    priority: 10
  - name: block-delete
    condition: {field: action, operator: eq, value: delete_account}
    action: deny
    priority: 100
defaults:
  action: deny
json
[
  {"name": "status allowed", "input": {"action": "order_status"}, "expected_allowed": true},
  {"name": "refund allowed", "input": {"action": "refund", "amount": 40}, "expected_allowed": true},
  {"name": "delete blocked", "input": {"action": "delete_account"}, "expected_allowed": false},
  {"name": "unknown tool blocked by default", "input": {"action": "wire_transfer"}, "expected_allowed": false}
]

Running the suite

agt test replays each fixture against the policy and reports a verdict for every one. A clean suite exits zero, which is what a CI step checks.

ExampleEvery rule in the shop policy has a fixture
agt test policy.yaml fixtures.json

Gating a pull request

In CI the exit code is what matters. This step fails the build the moment a fixture stops matching, so a change that drops the delete rule cannot merge. It is shown, not run here, because it belongs in a CI file.

yaml
# .github/workflows/policy.yml (a step)
- name: Test the governance policy
  run: agt test policy.yaml fixtures.json

Why a fixture per rule matters

  • A fixture that expects a block for every deny rule catches a rule that was dropped: the loader fails open, and only a block fixture turns that silent drop into a failure.
  • A default-deny fixture, the unknown tool here, proves the policy refuses what no rule names.
  • The suite is the policy's specification, so a reviewer reads the fixtures to see what the policy is meant to do.

agt test vs the running gate

agt testThe guard() gate
RunsIn CI, before mergeIn the app, on every call
ChecksFixtures match verdictsThe live tool call
CatchesA policy change that breaks a ruleAn attack at run time

Where you use the policy test

  • A required check on any pull request that touches policy.yaml.
  • A regression suite that grows a fixture every time a new rule is added.
  • A release gate, so a policy never ships without its fixtures passing.
Watch out. agt test uses the real loader, which silently drops a rule it does not understand. A suite with no block fixture for a rule can stay green while that rule quietly vanishes. Keep an expect-blocked fixture for every deny rule, so a dropped rule shows up as a mismatch.
Try it yourself
  • Delete the block-delete rule from the policy and run the suite to see the delete fixture fail.
  • Add a fixture for a refund over a limit once you add a rule that reads the amount.
  • Put both files in a directory and pass the directory to agt test.

Little by little, you're building something great.