Concepts
Severities
Creating a policy
1
Open Policies → Policies
Click Add policy.
2
Name and describe
The description is what reviewers see during an audit. Be specific.
3
Pick a severity
Critical and high contribute to the Compliance roll-up score; medium and low are reported but don’t block.
4
Define the evaluator
Built-in evaluators cover PII detection, toxicity, prompt-injection patterns, and a few model-specific checks. Custom evaluators are user-defined Python predicates.
5
Test against past runs
Use Evaluate to dry-run the policy against the last N runs of a chosen app. The preview shows which runs would have violated.
Bundling into packs
Policies travel in packs. A pack is what you assign — never an individual policy. Typical packs:- PII basics — emails, phone numbers, government IDs.
- EU AI Act high-risk — full set of obligations for high-risk apps.
- Internal data hygiene — no production data in development environments.
- Customer-facing assistant — refusal patterns, escalation triggers.
Compliance roll-up
The Compliance tab scores each app against the policies assigned to it. The score weights by severity — critical violations dominate, low violations contribute proportionally.score— 0–100violations_by_severitytop_violated_policies— useful for prioritising remediation
API
Related resources
Evidence
Capture the artefacts that prove policy enforcement.
Risk Dashboard
See how policy compliance feeds the per-app risk score.