1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
51 small wins to finish your pathNext question →
Your compliance team (or a regulator) asks you to demonstrate that your AI system is safe and reliable. What evidence do you prepare?
30-second answerSay your answer out loud first, then reveal.
Evidence package outline
| Section | Contents |
|---|---|
| System card | Intended use, out-of-scope uses, users, models and versions, data sources |
| Architecture and data flow | Diagram; where data is stored and processed; third parties; residency |
| Risk assessment | Harms identified (hallucination, bias, data leakage, misuse), likelihood and impact, mitigations |
| Evaluation report | Datasets (size, sourcing, representativeness), metrics, judge calibration, results by segment, CIs |
| Safety testing | Red-team scope, attack categories, success rates before and after fixes |
| Controls | Guardrails, permissions, HITL points, approval flows, kill switches |
| Monitoring | Online metrics, alert thresholds, review sampling, incident response runbook |
| Change management | Versioning, eval gates, sign-off process, rollback |
| Privacy | PII handling, retention, consent, DPIA |
| Limitations | Known failure modes, residual risks, user-facing disclosures |
Principles
- Reproducibility: pinned versions, dataset snapshots, scripts to rerun the evals.
- Honesty about limitations builds more trust than claims of perfection.
- Map to the relevant frameworks the organisation uses (internal model risk policy, NIST AI RMF, ISO/IEC 42001, sector guidance), in coordination with compliance.
Related
This is what real progress feels like.