Agent Governance ToolkitAgent Governance Toolkit 4.1.0 · Python 3.10+
Dashboard
0%

Risk scores

A risk score is a number from 0 to 1 the engine can compare against a threshold, but only one of RiskPolicy's three thresholds is ever read, so the other two cannot cap anything.

Last updated: 14 Oct, 2026 · Agent Governance Toolkit 4.1.0

The Context that changes at runtime lesson read runtime state to decide a call. A risk score is runtime state too: a number a scorer assigns to a request. RiskPolicy looks like the place to set limits on it, and two of its three fields read like hard caps. In 4.1.0 those two are stored and never read. This lesson proves it and shows the field that does work.

The fields that read like caps

RiskPolicy carries max_risk_score, require_approval_above and deny_above. The first two name thresholds you would expect to cap or gate a request. Set all three tight and register the policy on the engine.

python
import datetime

from agent_control_plane import PolicyEngine
from agent_control_plane.policy_engine import RiskPolicy
from agent_control_plane.agent_kernel import ExecutionRequest, AgentContext, ActionType

def make_request(amount):                        # the risk path works on a request object
    ctx = AgentContext(agent_id="support", session_id="s",
                       created_at=datetime.datetime.now())
    return ExecutionRequest(request_id="r", agent_context=ctx,
                            action_type=ActionType.API_CALL,
                            parameters={"amount": amount},
                            timestamp=datetime.datetime.now())

A tight cap that does nothing

Set max_risk_score and require_approval_above to 0.1, far below a request scored 0.8, and leave deny_above at 1.0. The 0.8 request should be far over the line. validate_risk allows it.

ExampleA 0.8 request sails through a 0.1 cap
policy = PolicyEngine()
policy.set_risk_policy("refunds",
    RiskPolicy(max_risk_score=0.1, require_approval_above=0.1, deny_above=1.0))

# 0.8 is far above the 0.1 these two fields name, yet the request passes
print("0.8 vs 0.1 caps:", policy.validate_risk(make_request(4000), 0.8))

Only deny_above is consulted

The one threshold the engine reads is deny_above. Set it to 0.5 and the 0.8 request is refused, while a 0.3 request passes. This is the real knob, and it is a single hard line, not a cap-plus-approval ladder.

Exampledeny_above draws the only line the engine honours
policy = PolicyEngine()
policy.set_risk_policy("refunds", RiskPolicy(deny_above=0.5))   # the one field read

# 0.8 is above deny_above and refused; 0.3 is below it and passes
print("score 0.8:", policy.validate_risk(make_request(4000), 0.8))
print("score 0.3:", policy.validate_risk(make_request(40), 0.3))

The source settles it

The reason the caps do nothing is visible in the library itself. Grep the installed packages for the three field names: max_risk_score and require_approval_above appear only once each, at their definition. deny_above appears twice, once at its definition and once where validate_risk compares against it.

$ grep -rn "max_risk_score" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:215:    max_risk_score: float = 0.5

$ grep -rn "require_approval_above" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:216:    require_approval_above: float = 0.7

$ grep -rn "deny_above" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:217:    deny_above: float = 0.9
agent_control_plane/policy_engine.py:595:            if risk_score >= policy.deny_above:

Which RiskPolicy field does what

FieldDefaultRead by the engine
max_risk_score0.5No, stored and never read
require_approval_above0.7No, stored and never read
deny_above0.9Yes, in validate_risk

How to cap risk so it holds

  • Use deny_above for a hard ceiling. It is the one threshold the engine compares, so a score at or above it is refused.
  • Treat max_risk_score and require_approval_above as not implemented. Setting them signals intent to a reader and changes no behaviour, so never rely on them to block or to gate for approval.
  • For anything deny_above cannot express, such as an approval step or a cap on the amount itself, write a rule that reads the value, which Writing your own policy rules and the argument conditions earlier in this part both do.

Where a risk score fits

  • A scorer that rates a request, feeding one number into deny_above as a last line before a dangerous call.
  • A coarse kill-switch, where any request a model or heuristic flags past a line is refused outright.
  • A signal to log, where the score is recorded for an audit even when deny_above lets the call through.
Watch out. The approval step require_approval_above implies does not exist here, the same way escalate on the first gate was stored and ignored. If a design calls for human approval above a score, build it yourself and do not assume RiskPolicy provides it.
Try it yourself
  • Set deny_above=0.9 and find the exact score where the verdict flips.
  • Add a high_risk_patterns entry to the policy and see it block a request whose parameters contain the pattern.
  • Print validate_risk for a score equal to deny_above and confirm the comparison is inclusive.

Slow is fine. Stopping is the only problem.