Risk scores
A risk score is a number from 0 to 1 the engine can compare against a threshold, but only one of RiskPolicy's three thresholds is ever read, so the other two cannot cap anything.
Last updated: 14 Oct, 2026 · Agent Governance Toolkit 4.1.0
The Context that changes at runtime lesson read runtime state to decide a call. A risk score is runtime state too: a number a scorer assigns to a request. RiskPolicy looks like the place to set limits on it, and two of its three fields read like hard caps. In 4.1.0 those two are stored and never read. This lesson proves it and shows the field that does work.
The fields that read like caps
RiskPolicy carries max_risk_score, require_approval_above and deny_above. The first two name thresholds you would expect to cap or gate a request. Set all three tight and register the policy on the engine.
import datetime
from agent_control_plane import PolicyEngine
from agent_control_plane.policy_engine import RiskPolicy
from agent_control_plane.agent_kernel import ExecutionRequest, AgentContext, ActionType
def make_request(amount): # the risk path works on a request object
ctx = AgentContext(agent_id="support", session_id="s",
created_at=datetime.datetime.now())
return ExecutionRequest(request_id="r", agent_context=ctx,
action_type=ActionType.API_CALL,
parameters={"amount": amount},
timestamp=datetime.datetime.now())A tight cap that does nothing
Set max_risk_score and require_approval_above to 0.1, far below a request scored 0.8, and leave deny_above at 1.0. The 0.8 request should be far over the line. validate_risk allows it.
policy = PolicyEngine()
policy.set_risk_policy("refunds",
RiskPolicy(max_risk_score=0.1, require_approval_above=0.1, deny_above=1.0))
# 0.8 is far above the 0.1 these two fields name, yet the request passes
print("0.8 vs 0.1 caps:", policy.validate_risk(make_request(4000), 0.8))0.8 vs 0.1 caps: True
Only deny_above is consulted
The one threshold the engine reads is deny_above. Set it to 0.5 and the 0.8 request is refused, while a 0.3 request passes. This is the real knob, and it is a single hard line, not a cap-plus-approval ladder.
policy = PolicyEngine()
policy.set_risk_policy("refunds", RiskPolicy(deny_above=0.5)) # the one field read
# 0.8 is above deny_above and refused; 0.3 is below it and passes
print("score 0.8:", policy.validate_risk(make_request(4000), 0.8))
print("score 0.3:", policy.validate_risk(make_request(40), 0.3))score 0.8: False score 0.3: True
The source settles it
The reason the caps do nothing is visible in the library itself. Grep the installed packages for the three field names: max_risk_score and require_approval_above appear only once each, at their definition. deny_above appears twice, once at its definition and once where validate_risk compares against it.
$ grep -rn "max_risk_score" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:215: max_risk_score: float = 0.5
$ grep -rn "require_approval_above" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:216: require_approval_above: float = 0.7
$ grep -rn "deny_above" agent_control_plane agent_os agent_runtime
agent_control_plane/policy_engine.py:217: deny_above: float = 0.9
agent_control_plane/policy_engine.py:595: if risk_score >= policy.deny_above:Which RiskPolicy field does what
| Field | Default | Read by the engine |
|---|---|---|
max_risk_score | 0.5 | No, stored and never read |
require_approval_above | 0.7 | No, stored and never read |
deny_above | 0.9 | Yes, in validate_risk |
How to cap risk so it holds
- Use
deny_abovefor a hard ceiling. It is the one threshold the engine compares, so a score at or above it is refused. - Treat
max_risk_scoreandrequire_approval_aboveas not implemented. Setting them signals intent to a reader and changes no behaviour, so never rely on them to block or to gate for approval. - For anything
deny_abovecannot express, such as an approval step or a cap on the amount itself, write a rule that reads the value, which Writing your own policy rules and the argument conditions earlier in this part both do.
Where a risk score fits
- A scorer that rates a request, feeding one number into
deny_aboveas a last line before a dangerous call. - A coarse kill-switch, where any request a model or heuristic flags past a line is refused outright.
- A signal to log, where the score is recorded for an audit even when
deny_abovelets the call through.
require_approval_above implies does not exist here, the same way escalate on the first gate was stored and ignored. If a design calls for human approval above a score, build it yourself and do not assume RiskPolicy provides it.Related
- Previous: Context that changes at runtime
- Next: Where the audit trail is written
- See also: govern(): the smallest real gate
- Reference: Agent Governance Toolkit documentation
- Set
deny_above=0.9and find the exact score where the verdict flips. - Add a
high_risk_patternsentry to the policy and see it block a request whose parameters contain the pattern. - Print
validate_riskfor a score equal todeny_aboveand confirm the comparison is inclusive.
Slow is fine. Stopping is the only problem.