Healthcare
Clinical Imaging Triage & Patient Follow-Up
High RiskUnverifiedDiscuss / dispute
Diagnostic imaging pipelines monitored for critical findings, then follow-up scheduling, lab orders, record updates and draft patient notifications executed in the EHR — held until the supervising physician confirms.
Classification rationale: Annex III(5) and the medical-device route apply: a triage system influencing access to care is high-risk under the AI Act and simultaneously a device under MDR/IVDR, so conformity assessment and clinical evaluation stack on top of the AI Act duties. Art. 14 is satisfied only if dispatch is physically gated on the clinician's confirmation.
Indicative decision support, not legal advice. Risk classification depends on your concrete deployment context and can change with scope drift — validate the result with qualified counsel.
pricingPer completed patient-care workflow / per resolved follow-up
oversightRadiologist or attending physician confirms the proposed care plan before any patient-facing dispatch
Compliance brief
This use case is high-risk under the EU AI Act (High Risk); the provider and deployer obligations apply in full.
What is owed
- Art. 9. Continuous, iterative risk-management system across the whole lifecycle: identify, estimate, evaluate, mitigate; testing incl.
- Art. 10. Quality criteria for training/validation/test data: relevance, representativeness, error-freeness, bias detection & mitigation, data-governance procedures.
- Art. 11. Annex IV technical file before placing on market: system description, architecture, capabilities/limitations, risk measures — kept up to date.
- Art. 12. Automatic, tamper-evident event logging over the system lifetime, serving three regulatory objectives: risk identification (Art.
- Art. 13. Instructions for use: capabilities, limitations, intended purpose, human-oversight measures, expected accuracy.
Dates that bind
- 2026-08-02 — General applicability + Art. 50. Transparency obligations for chatbots, deepfakes and synthetic content; EU-level enforcement begins.
- 2026-12-02 — Additional prohibitions. Additional bans (deepfake CSAM et al.) and transition period for synthetic content under Art. 50(2).
Maximum exposure
- EU AI Act: Tiered: €35m / 7% (prohibited practices); €15m / 3% (Art. 9–15 high-risk obligations incl. data governance, documentation, logging); €7.5m / 1% (Art. 99(5) — incorrect, incomplete or misleading information to notified bodies or national competent authorities)
- GDPR: Up to €20m or 4% of worldwide annual turnover
First five actions
- Confirm in writing whether this organisation builds/places the system on the market (provider) or only operates it (deployer), since the role is not yet established.
- Commission and confirm the Art. 9, Art. 10, Art. 11 obligations named above as active workstreams with an accountable owner.
- Stand up the named oversight design — Mode 1 — with a documented human-review procedure.
- Produce the technical documentation and evidence artefacts already mapped to this use case (Live Risk Register / Posture Management, Watchdog Supervisor & Rate Limiting, Deterministic Policy Engine (OPA / Cedar)) before they are requested.
- Put 2026-08-02 — General applicability + Art. 50 — into the compliance calendar with an owner and lead time.
Terms used above: · · ·
Applicable Regulations (10)
EU AI Act (Regulation (EU) 2024/1689)
unverified · no verification date source EUR-Lex
Horizontal, risk-based product-safety law for AI systems and GPAI models. Extraterritorial market-place principle. Staged applicability 2025–2030 (Digital Omnibus: Annex III → 2 Dec 2027, Annex I → 2 Aug 2028).
Sanctions: Tiered: €35m / 7% (prohibited practices); €15m / 3% (Art. 9–15 high-risk obligations incl. data governance, documentation, logging); €7.5m / 1% (Art. 99(5) — incorrect, incomplete or misleading information to notified bodies or national competent authorities)
MDR / IVDR (Regulations (EU) 2017/745 & 2017/746)
unverified · no verification date source EUR-Lex
Medical-device and IVD regulation. AI as (part of) a medical device makes the system Annex I high-risk under the AI Act; notified-body conformity assessment.
GDPR (Regulation (EU) 2016/679)
unverified · no verification date source EUR-Lex
Applies unchanged next to the AI Act for all personal data in training, fine-tuning, RAG and inference. Key friction points: Art. 22 automated decisions, Art. 17 erasure vs. AI Act logging, Art. 35 DPIA.
Sanctions: Up to €20m or 4% of worldwide annual turnover
HIPAA (US Health Privacy)
in-force · verified 2026-08-04
US health-data regime (Privacy, Security & Breach Notification Rules): PHI minimum-necessary standard, BAA chains for AI vendors, audit controls and access logging. For clinical AI (scribing, diagnostics, prior-auth) it is the US-side twin of GDPR Art. 9 — evidence overlap: access logs, vendor due diligence, encryption attestation.
EHDS (European Health Data Space Regulation)
unverified · no verification date
Primary and secondary use of electronic health data; access via health-data access bodies for AI training.
Art. 14 — Human Oversight
unverified · no verification date source artificialintelligenceact.eu
Effective human oversight by qualified, trained natural persons with real authority to intervene, override and stop; interface duties (interpretability, automation-bias countermeasures); oversight must be commensurate with autonomy level. Competence and authority of the overseers is itself a testable control objective — training records and oversight protocols are its evidence.
Art. 9 — Risk Management
unverified · no verification date source artificialintelligenceact.eu
Continuous, iterative risk-management system across the whole lifecycle: identify, estimate, evaluate, mitigate; testing incl. against misuse.
Art. 13 — Transparency to Deployers
unverified · no verification date source artificialintelligenceact.eu
Instructions for use: capabilities, limitations, intended purpose, human-oversight measures, expected accuracy.
Art. 26 — Deployer Obligations
unverified · no verification date source artificialintelligenceact.eu
Use per instructions, assign competent human oversight, input-data control, log retention ≥ 6 months, inform workers, incident duty.
Art. 43 — Conformity Assessment
unverified · no verification date source artificialintelligenceact.eu
Two pathways: internal control self-assessment (Annex VI) for most Annex III systems (HR, credit, education) — provider verifies QMS (Art. 17), technical file (Annex IV) and design consistency; notified-body assessment (Annex VII) mandatory for safety components in harmonised products (medical devices, aviation, rail) and remote biometric identification. Harmonised standards in the OJEU give presumption of conformity.
Legal Obligations (17)
Art. 9 — Risk Management
Continuous, iterative risk-management system across the whole lifecycle: identify, estimate, evaluate, mitigate; testing incl. against misuse.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 10 — Data Governance
Quality criteria for training/validation/test data: relevance, representativeness, error-freeness, bias detection & mitigation, data-governance procedures.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 11 — Technical Documentation
Annex IV technical file before placing on market: system description, architecture, capabilities/limitations, risk measures — kept up to date.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 12 — Record-Keeping / Logging
Automatic, tamper-evident event logging over the system lifetime, serving three regulatory objectives: risk identification (Art. 79), post-market monitoring (Art. 72) and deployer oversight (Art. 26(5)). Deployers retain logs ≥ 6 months; financial institutions fold them into statutory internal audit documentation. A bolted-on logging wrapper does not satisfy the requirement — logging must be core architecture.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 13 — Transparency to Deployers
Instructions for use: capabilities, limitations, intended purpose, human-oversight measures, expected accuracy.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 14 — Human Oversight
Effective human oversight by qualified, trained natural persons with real authority to intervene, override and stop; interface duties (interpretability, automation-bias countermeasures); oversight must be commensurate with autonomy level. Competence and authority of the overseers is itself a testable control objective — training records and oversight protocols are its evidence.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 15 — Accuracy, Robustness, Cybersecurity
Appropriate accuracy levels, resilience against errors and adversarial attacks (data poisoning, evasion, prompt injection), declared metrics.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 17 — Quality Management System
Product-focused QMS for providers: strategy, design controls, data management, post-market monitoring — target of EN 18286:2026, the first AI Act harmonised-standard candidate to be published; presumption of conformity applies only once it is cited in the OJEU, which is still pending.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 43 — Conformity Assessment
Two pathways: internal control self-assessment (Annex VI) for most Annex III systems (HR, credit, education) — provider verifies QMS (Art. 17), technical file (Annex IV) and design consistency; notified-body assessment (Annex VII) mandatory for safety components in harmonised products (medical devices, aviation, rail) and remote biometric identification. Harmonised standards in the OJEU give presumption of conformity.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 72/73 — Post-Market Monitoring & Incidents
Post-market monitoring plan and serious-incident reporting (15 days; 2 days for widespread infringement) to market-surveillance authorities.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 26 — Deployer Obligations
Use per instructions, assign competent human oversight, input-data control, log retention ≥ 6 months, inform workers, incident duty.
unverified · no verification date read the article artificialintelligenceact.eu
Art. 47/48 — CE Marking & Declaration of Conformity
After passing conformity assessment: written EU Declaration of Conformity (retained 10 years) and visible, indelible CE marking — for digital systems a digital CE mark accessible via UI or machine-readable code; notified-body number displayed where one was involved.
unverified · no verification date read the article artificialintelligenceact.eu
GDPR Art. 22 — Automated Decisions
Right not to be subject to solely automated decisions with legal/similar effect; requires meaningful human involvement or explicit legal basis + safeguards.
unverified · no verification date read the article EUR-Lex
GDPR Art. 17 — Erasure
Right to erasure collides with AI Act Art. 12 immutable logging — resolved architecturally via bitemporal data modelling + physical partition scrub.
unverified · no verification date read the article EUR-Lex
GDPR Art. 25 — Data Protection by Design
Privacy by design & default: minimisation, pseudonymisation, PII filters in pipelines and vector stores.
unverified · no verification date read the article EUR-Lex
GDPR Art. 35 — DPIA
Data-protection impact assessment for high-risk processing — pairs with AI Act fundamental-rights impact assessment (Art. 27) for public-facing high-risk systems.
unverified · no verification date read the article EUR-Lex
Art. 25 — Value Chain / Role Flip
A deployer becomes the provider (full Art. 8–17 duties) by re-branding, changing intended purpose, or making a substantial modification — e.g. deep fine-tuning or wiring a model into autonomous agent toolchains.
unverified · no verification date read the article artificialintelligenceact.eu
Control Objectives (11)
obligation (article) → operationalized_by → control objective → satisfied_by → component/pattern; control objective → evidenced_by → evidence artifact
Art. 9
Pre-Deployment AI Risk & Impact Assessment
ISO 42001 Clauses 6.1 / 8.2: every workflow is risk-assessed and impact-assessed before production, covering fundamental rights, availability, privacy and legal liability.
ISO/IEC 42001 clause 8.2 (indicative)
satisfied by: AI Register & Model Registry / Factsheets
evidenced by: AI Impact Assessment (AIIA)
Art. 10
Data Quality & Origin Management
ISO 42001 Annex A.7: training, fine-tuning and RAG corpora are audited for lineage, quality, representativeness and IP clearance.
ISO/IEC 42001 clause A.7 (indicative)
satisfied by: Data Lineage & Versioning
evidenced by: Dataset Documentation & Bias Reports
Art. 11
control layer: community mandate — propose objectives
Art. 12
Log Completeness & Coverage
Every lifecycle-relevant event is captured with the FprEN ISO/IEC 24970 field set (input/output traces, timestamps, acting identity, referenced sources, human overrides) across all execution paths — including tool calls, retries and degraded modes. Testable: trace-coverage sampling against the event taxonomy; gaps are findings.
satisfied by: OpenTelemetry / FCoT TracingWORM / Immutable Audit Vault
evidenced by: Event Logs & Decision Traces
Log Integrity & Non-Repudiation
Recorded events cannot be altered, reordered or deleted without detection — including by administrators of the log store. Testable: hash-chain verification, external-anchor reconciliation, red-team attempt to rewrite history. This is the control objective the WORM vault alone does not fully satisfy.
evidenced by: Event Logs & Decision Traces · Immutable Decision Ledger (WORM)
Log Access & Retention Governance
Logs are retained per regime (deployer ≥ 6 months, financial institutions per audit rules, provider lifetime scopes), access is least-privilege and itself logged, and lawful disclosure (authority request, PLD disclosure order) is executable without exposing unrelated data. Testable: retention audit, access-review, disclosure fire drill.
satisfied by: Central Credential VaultBitemporal Memory (GDPR×Art.12)
evidenced by: Human-Oversight Protocol & Intervention Records
Art. 13
System Traceability & Decision Transparency
ISO 42001 Annex A.8: documented mechanisms explain how a given decision was produced, enabling root-cause analysis after an incident.
ISO/IEC 42001 clause A.8 (indicative)
satisfied by: OpenTelemetry / FCoT Tracing
evidenced by: AI System Model Card
Art. 14
Oversight Competence & Authority
The natural persons overseeing the system are trained for it, have documented authority to intervene/override/stop, actually exercise it (override rates monitored against automation bias), and their competence is refreshed on system change. Testable: training records, oversight-protocol drills, override statistics.
ISO/IEC 42001 clause 7.2 (indicative)
satisfied by: HITL Escalation Queue & Review UI
evidenced by: AI Literacy Training Records · Human-Oversight Protocol & Intervention Records · Human Oversight Operating Standard (SOP)
Art. 15
Adversarial Robustness Verified
Resilience against evasion, poisoning and extraction is demonstrated against the declared threat model at a defined cadence, with metrics per ISO/IEC 4213 / 24029 and regressions gated. Testable: red-team reports with reproducible probes; a passed probe from last year is not evidence for this year's model.
satisfied by: Bias Testing & Data Quality Pipeline
evidenced by: Accuracy, Robustness & Red-Teaming Reports
Runtime Injection Defense
Direct and indirect prompt injection attempts are detected, contained or neutralized at runtime across all ingestion paths (user input, retrieved documents, tool outputs, inter-agent messages). Testable: guardrail telemetry with measured catch rates against a maintained injection corpus; command/data separation enforced structurally, not by instruction.
satisfied by: Input Rails / Prompt ShieldsGuardrail Sidecar / InterceptionTrinity Defense (TCB + Command Gates + IFC)
evidenced by: Guardrail Telemetry & Sanitization Records
Art. 17
control layer: community mandate — propose objectives
Art. 43
control layer: community mandate — propose objectives
Art. 72/73
Performance Monitoring & Drift Management
ISO 42001 Clauses 9 & 10: continuous accuracy, drift and incident monitoring feeding internal audit and recertification.
ISO/IEC 42001 clause 9.1 (indicative)
satisfied by: Model Drift & Accuracy Monitor
evidenced by: Post-Market Monitoring Plan & Incident Reports
Art. 26
Embedded-AI Vendor Governance
Every third-party system with embedded AI (closed SaaS, productized services, model APIs) is inventoried, risk-tiered and governed by contract and attestation — because inline runtime inspection is architecturally impossible in a vendor's closed execution path. Required: intake-based vendor risk workflow, contractual ZDR & non-training clauses, AI-BOM/Factsheet disclosure, attestation review cadence, exit strategy. Testable: sample vendor list vs. register completeness; check contracts for AI clauses; verify attestations current.
ISO/IEC 42001 clause A.10 (indicative)
evidenced by: Vendor & Model Due-Diligence Records · C5 / AIC4 Cloud Attestation · AI Bill of Materials (AI-BOM) & Factsheets
Art. 47/48
control layer: community mandate — propose objectives
GDPR Art. 22
control layer: community mandate — propose objectives
GDPR Art. 17
control layer: community mandate — propose objectives
GDPR Art. 25
control layer: community mandate — propose objectives
GDPR Art. 35
control layer: community mandate — propose objectives
Art. 25
control layer: community mandate — propose objectives
Take this into your GRC tooling
A control mapping your ISO/IEC 42001 or CSA AICM workbook can ingest, and an Annex IV skeleton to start the technical file from. Indicative mappings only — cells we are not confident about are exported empty rather than filled in.
Standards & Evidence
ISO/IEC 23894 (AI Risk Management)
AI risk-management guidance extending ISO 31000 — feeds the Art. 9 risk-management system.
unverified · no verification date publisher ISO
evidence for: Art. 9 · Art. 9 — Risk Management
NIST AI RMF 1.0
Govern–Map–Measure–Manage risk framework; Map/Measure functions populate the Art. 9 risk register with quantified values; the transatlantic mapping reference.
unverified · no verification date
evidence for: Art. 9 · Art. 9 — Risk Management
JTC 21 Technical Package (prEN 18228/18229/18281–83)
CEN-CENELEC JTC 21 technical package under standardisation request M/593 (prEN 18228 trustworthiness, 18229 risk management, 18281–83 CV/NLP evaluation et al.); staged drafts, none OJEU-cited yet — Annex III applicability (Dec 2027) is Omnibus-coupled to their availability.
draft · verified 2026-08-04
evidence for: Art. 9 · Art. 10 · Art. 9 — Risk Management
FAIR-AIR / FAIR-MAM
AI extension of Factor Analysis of Information Risk: Expected Financial Loss = Loss Event Frequency (threat frequency × vulnerability) × Loss Magnitude (primary + secondary), run as Monte Carlo distributions — the standard bridge from technical AI failure modes to board-level monetary exposure.
unverified · no verification date
evidence for: Art. 9 · Art. 9 — Risk Management
NIST AI 600-1 (GenAI Profile)
Companion profile to the AI RMF covering twelve GenAI-specific failure modes — confabulation, prompt injection, value-chain propagation and others — as the technical checklist behind Map/Measure for generative systems.
unverified · no verification date
evidence for: Art. 9 · Art. 15 · Art. 9 — Risk Management
Gartner AI TRiSM
AI Trust, Risk and Security Management — industry framework formalizing continuous AI oversight across four pillars: governance (inventory, AI-BOM, decision rights, change approval), trustworthiness & fairness (explainability, bias), reliability (drift, hallucination metrics), security management (prompt injection, model inversion, poisoning, leakage). No legal force; its value is the architecture it implies — the four-layer enterprise stack and the first/second-line separation this graph models as bp-trism and pat-lines-defense.
published · verified 2026-08-06
evidence for: Art. 9 · Art. 9 — Risk Management
ISO/IEC 5259 (Data Quality for ML)
Five-part data-quality framework (governance, process, management) — direct evidence path for Art. 10 representativeness and completeness.
unverified · no verification date publisher ISO
evidence for: Art. 10
EN ISO/IEC 22989 (AI Concepts)
Published terminology and concepts standard — the shared vocabulary layer for documentation and audits.
unverified · no verification date publisher ISO
evidence for: Art. 11
FprEN ISO/IEC 24970 (AI Logging)
Specifies event logging in AI systems — the concrete implementation target for Art. 12 record-keeping.
formal-vote · verified 2026-08-04 publisher ISO
evidence for: Art. 12
TAGOF (Audit-as-Code)
Operationalizes governance as code in CI/CD: policy-as-code enforcement, continuous runtime telemetry and automatically generated audit evidence — the execution layer that replaces periodic audits with continuous assurance.
unverified · no verification date
evidence for: Art. 12
prEN ISO/IEC 12792 (Transparency Taxonomy)
Transparency taxonomy for AI systems: structured disclosure of system composition, data provenance, capabilities and limitations. The harmonised-norm candidate backing Art. 13 instructions-for-use and deployer-information duties — defines what a complete transparency package must contain.
draft · verified 2026-08-04 publisher ISO
evidence for: Art. 13 · Art. 13 — Transparency to Deployers
OWASP Agentic Security (AST10 / Core Risks)
Threat framework for autonomous agents: tool misuse, excessive agency, confused-deputy, memory poisoning — with AIVSS scoring.
unverified · no verification date
evidence for: Art. 14 · Art. 14 — Human Oversight
IMDA Agentic AI Governance Framework (SG)
Model AI Governance Framework for Agentic AI (Jan 2026, updated Jun 2026) — first state-issued agentic-specific guidance: bounded autonomy levels, action-space and interface restrictions, human-in-command checkpoints, automation-bias controls, logging & attribution expectations. No legal force in the EU, but the most concrete public benchmark for Art. 14-style oversight design of agent systems.
published · verified 2026-08-04
evidence for: Art. 14 · Art. 14 — Human Oversight
DIN SPEC 92001-1/-2/-3
AI life-cycle quality metamodel: functionality, robustness (adversarial & corruption), traceability/explainability — German operationalisation for Art. 15.
unverified · no verification date
evidence for: Art. 15
ISO/IEC 24029 (NN Robustness)
Robustness assessment of neural networks incl. formal methods (part 2) — supports Art. 15 evidence.
unverified · no verification date publisher ISO
evidence for: Art. 15
OWASP Top 10 for LLM Apps (2025)
The de-facto technical security standard for GenAI applications; maps to Art. 10/14/15 and ISO 42001 Annex A controls.
unverified · no verification date
evidence for: Art. 15
ENISA Multilayer Framework & AI Threat Landscape
Three-layer good-practice model (cyber foundations → AI-specific → sectoral) and lifecycle threat landscape — the operational base for Art. 15 and CRA.
unverified · no verification date
evidence for: Art. 15
BSI C5:2026
Cloud compliance catalogue (168 requirements): post-quantum crypto, confidential computing, container security — infrastructure evidence layer for NIS2/CRA/Art. 15; binding baseline from June 2027.
unverified · no verification date
evidence for: Art. 15
ISO/IEC TS 4213 (ML Performance Measurement)
Assessment of machine-learning classification performance: standardized metrics, test-set discipline, reporting format. The metric backbone for Art. 15 'declared accuracy' — test reports that cite it are comparable across vendors and audits.
published · verified 2026-08-04 publisher ISO
evidence for: Art. 15
BSI GenAI Criteria Catalogue
Criteria for integrating external generative models via API: named AI owner, central AI register, case-by-case risk analysis, multi-stage input/output validation, least privilege, prompt/permission separation.
unverified · no verification date
evidence for: Art. 15
EN 18286:2026 (QMS for AI Act)
Harmonised-norm candidate translating Art. 17 QMS into a product-focused governance framework; mappings to ISO 9001 and ISO/IEC 42001 Annex A (Annexes C & D); published as EN 18286:2026 in July 2026, OJEU citation (and with it the presumption of conformity) still pending.
published · verified 2026-08-04
evidence for: Art. 17
ISO/IEC 42001:2023 (AIMS)
ISO/IEC 42001:2023 — certifiable AI management system (Annex SL harmonized structure, PDCA logic, synergy discount when an ISO 27001 ISMS exists). Clauses 4–10 plus Annex A controls (control count 38 vs 39 is a live community dispute — counting method differs by edition/guide). Covers an estimated 40–50% of AI Act organizational duties; organizational certificate, no product presumption of conformity.
published · verified 2026-08-04 publisher ISO
evidence for: Art. 17
ISO/IEC 42006 (Audit Bodies)
Requirements for bodies auditing/certifying AIMS — accreditation basis (e.g. DAkkS) for ISO 42001 certificates.
unverified · no verification date publisher ISO
evidence for: Art. 43 · Art. 43 — Conformity Assessment
ISO/IEC 42005 (AI Impact Assessment)
Guidance for AI system impact assessments — supports DPIA/FRIA-style analyses.
unverified · no verification date publisher ISO
evidence for: GDPR Art. 35
IEEE CertifAIEd™
Ethics certification (transparency, accountability, algorithmic bias, privacy) for products and professionals; interfaces with the EU ALTAI assessment list.
unverified · no verification date
evidence for: EU AI Act
Architecture Blueprint
Human-in-the-Loop Core Pattern
For high-risk decision support over people (HR, credit, benefits): confidence-thresholded escalation queues, explainability API (SHAP/LIME or CoT traces), WORM audit vault, bias testing per ISO 5259 — the human decision is architecturally enforced.
Mode 1 — Assistant (HITL)
Agent proposes, human disposes: every consequential action reviewed before execution. Default for first deployments, irreversible or legally sensitive actions.
Required Technical Components (32)
Live Risk Register / Posture Management
Continuously updated risk register wired to runtime posture: threat-model deltas, open defects, control status, exposure per system. Includes Shadow-AI discovery — continuous scanning for unsanctioned agents, MCP servers and AI API usage outside the register; an unregistered agent is an unmanaged Art. 12/26 liability and the empirical driver of proportionate (not blanket) controls.
from: Art. 9 · GDPR Art. 35
Watchdog Supervisor & Rate Limiting
Cost/iteration caps, loop detection, anomaly-triggered mandatory approval (CodeBuddy 'suspicious command override').
from: Art. 9 · Art. 15
Deterministic Policy Engine (OPA / Cedar)
Policy-as-code decision point (PDP) with enforcement points (PEP) in front of every tool call: versioned policies in Git, microsecond evaluation, typed action schemas — authorization decided outside the model's reasoning space, never in the prompt.
from: Art. 9
Guardian Agents (Runtime Policy Enforcement)
Autonomous supervisory agents outside the supervised agent's reasoning loop: stateful threat engines with graph-based cross-session history (catch multi-turn injection, gradual exfiltration, incremental privilege escalation), event-driven exposure visibility (permission drift, new connectors), and contextual risk correlation into unified issues — interception before execution, not post-hoc logging.
from: Art. 9 · Art. 72/73
Bias Testing & Data Quality Pipeline
Representativeness checks, bias metrics and mitigation per ISO/IEC 5259; versioned datasets with lineage.
from: Art. 10 · Human-in-the-Loop Core Pattern
Data Lineage & Versioning
Provenance tracking of datasets, features and embeddings; write-time attribution (source, actor, timestamp, confidence).
from: Art. 10
PII Scrubbing / DLP-NER Layer
Automated detection, pseudonymisation and blocking of personal data in inputs, retrievals and outputs.
from: Art. 10 · GDPR Art. 25
Retrieval Rails (ACL-aware RAG)
Relevance, freshness and per-user permission checks on every retrieved chunk; curated, versioned index.
from: Art. 10
Sovereign Context Layer
Governed runtime workspace operationalizing Art. 10: traceable lineage for every RAG chunk and training record at execution time, canonical version-controlled business glossary (documents Art. 10(2)(d) baseline assumptions), and continuous data-quality monitoring with threshold alerts and logged remediation for the Art. 10(3) 'error-free and complete' standard.
from: Art. 10
Inline PII/PHI Tokenisation
Personal and health data are detected and replaced with reversible cryptographic tokens before the payload leaves the isolation layer; re-identification happens only inside the tenant boundary.
from: Art. 10
AI Register & Model Registry / Factsheets
AI register & model registry: central inventory of every model, agent, RAG pipeline and embedded third-party SaaS AI across the estate, with factsheets per asset. v2.0 duty: every application — internal, open-source or procured — continuously publishes a machine-readable AI-BOM and Factsheet into the register; an asset without a current AI-BOM is an inventory gap, not a formality. Feeds Colorado AIA/ LL144 disclosure duties and the Art. 11 technical file; the enforcement backstop is Shadow-AI discovery on the risk register.
from: Art. 11 · Art. 13
WORM / Immutable Audit Vault
Append-only, hash-chained audit vault (WORM object-lock storage, AES-256 at rest, TLS 1.3 in transit). Guarantees tamper-evidence within the organization's trust domain — which stops your own team, but not an admin who can rebuild the vault. Pair with an external trust anchor and key ceremonies outside the operating team for evidence that holds against the insider scenario.
from: Art. 12 · Art. 26 · Human-in-the-Loop Core Pattern
OpenTelemetry / FCoT Tracing
Hierarchical trace spans for every sub-task, prompt, retrieved document and API call — the reconstructible decision path for Art. 12/14 and PLD disclosure.
from: Art. 12
External Trust Anchor (Qualified Timestamp / Ledger)
Takes integrity proofs out of the operator's trust domain: periodic anchoring of log hash-chain heads via qualified electronic timestamps or a (qualified) electronic ledger per eIDAS 2, with signing keys held outside the operating team (key ceremony, HSM, separation of duties). Answers the insider test — a party who controls the vault cannot rewrite history without the anchor exposing it. Cost profile: anchoring is periodic and cheap; it upgrades every downstream log-based artifact at once.
from: Art. 12
Explainability API (SHAP/LIME/CoT)
Feature attributions for classical ML, reasoning-trace summaries for GenAI — feeds the human reviewer and the technical file.
from: Art. 13 · Art. 14 · Human-in-the-Loop Core Pattern
HITL Escalation Queue & Review UI
HITL escalation queue & review UI ('Human-as-a-Tool': the agent calls the human like any other tool via propose-action objects). Confidence- and risk-threshold routing, SLA timers, structured accept/modify/reject verdicts with digital reviewer signature at gate release — each verdict is itself Art. 14 evidence and feeds the active-learning loop.
from: Art. 14 · Art. 26 · GDPR Art. 22 · Human-in-the-Loop Core Pattern
Kill Switch / Graceful Degradation
Operator stop controls and degraded-mode fallbacks; real-time override (veto) channels for HOTL operation.
from: Art. 14 · Human-in-the-Loop Core Pattern
Trust & Risk Dual Scoring
Escalation triggers built from two independent signals, because raw model confidence is uncalibrated: calibrated trust scores (prompt relevance, similarity to historic successes, cross-model consistency) plus deterministic risk scores (sensitive categories, transaction value, protected data) — either crossing its threshold forces human review.
from: Art. 14 · Human-in-the-Loop Core Pattern
Confidence-Threshold HITL Routing
Every output carries a confidence score C_s. C_s ≥ θ commits to the immutable ledger and downstream systems; C_s < θ pauses the transaction and routes the payload to a specialist review queue, whose verdict is logged as part of the decision record.
from: Art. 14
Confidence Scoring & Threshold Gate
Computes a probabilistic confidence score for every output and holds the transaction when the score falls below the workflow's regulatory threshold.
from: Art. 14 · Human-in-the-Loop Core Pattern
Input Rails / Prompt Shields
Pre-model validation of user input: injection detection, topic blocking, encoding checks.
from: Art. 15
Output Rails / Groundedness Check
Faithfulness scoring of answers against retrieved sources; deterministic fallback instead of hallucination; schema-validated structured output.
from: Art. 15
Confidential Computing Enclaves
AMD SEV / Intel TDX: data protected from the cloud operator even in memory during inference.
from: Art. 15
Model Abstraction & Graceful Fallback
Application logic addresses capabilities, not providers; the router degrades to a secondary or local model on error-rate or latency breach instead of failing the workflow.
from: Art. 15
AI Intake Portal & Use-Case Triage
The operational front door of the translational pipeline: structured intake profile (business objective, autonomy degree, data sensitivity, deployment context, target users) → automated tier proposal (detectors + evaluator pipeline) → risk-proportionate approval workflow → register entry with AI-BOM stub. Prevents both over-engineering (blanket high-tier controls breed Shadow AI) and under-engineering (unassessed high-risk deployment). Every governance framework assumes it; almost no failed audit had one.
from: Art. 17
Unified Incident-Response Runbook
One procedure reconciling AI Act Art. 73, GDPR Art. 33 (72h), DORA and NIS2 (24h/72h) timelines and recipients.
from: Art. 72/73
Bitemporal Memory (GDPR×Art.12)
valid_from/valid_to + transaction time on every record: GDPR erasure removes data from the active retrieval path while the HMAC-chained immutable log survives for Art. 12 / PLD defence; tenant-scoped partitions allow physical scrub of PII.
from: GDPR Art. 17
Per-Tenant Retrieval Segmentation
Retrieval is scoped by tenant and by caller entitlement at query time, preventing cross-client and cross-role leakage through shared indexes.
from: GDPR Art. 25
Segmented Vector Store (RBAC + CMEK)
Vector indexes, embeddings and document stores are logically and physically partitioned per client, with role-based access and customer-managed encryption keys.
from: GDPR Art. 25
Zero-Data-Retention Vendor Binding
Sensitive inference is contractually and technically restricted to endpoints under zero-data-retention and non-training terms, evidenced per vendor and re-validated annually.
from: Art. 25
Durable Checkpointing (Pause & Resume)
At oversight gates the complete operational state — working memory, conversation history, tool arguments, intermediate artifacts — is serialized into a durable checkpoint (fast KV store for sub-ms lookups, transactional backend as recovery anchor, vector store for semantic caching of past human decisions). On approval the agent deserializes and resumes at the exact step; matched precedents can shortcut re-planning entirely.
from: Human-in-the-Loop Core Pattern
Active-Learning Feedback Loop
Human corrections at oversight gates are serialized as structured data — original context, model proposal, human edit, rationale — and fed into fine-tuning pipelines and prompt registries, systematically reducing future escalation rates instead of dying in review UIs.
from: Human-in-the-Loop Core Pattern
Delivery Stack & Pipeline Stage (10)
Service-as-a-Software delivery: the engines, patterns and artifacts this workflow needs on top of the generic obligations. See the full pipeline
HITL Escalation Queue & Review UI
HITL escalation queue & review UI ('Human-as-a-Tool': the agent calls the human like any other tool via propose-action objects). Confidence- and risk-threshold routing, SLA timers, structured accept/modify/reject verdicts with digital reviewer signature at gate release — each verdict is itself Art. 14 evidence and feeds the active-learning loop.
pipeline stage 4 — Output audit & human-in-the-loop gateway
Confidence Scoring & Threshold Gate
Computes a probabilistic confidence score for every output and holds the transaction when the score falls below the workflow's regulatory threshold.
pipeline stage 4 — Output audit & human-in-the-loop gateway
Explainability API (SHAP/LIME/CoT)
Feature attributions for classical ML, reasoning-trace summaries for GenAI — feeds the human reviewer and the technical file.
WORM / Immutable Audit Vault
Append-only, hash-chained audit vault (WORM object-lock storage, AES-256 at rest, TLS 1.3 in transit). Guarantees tamper-evidence within the organization's trust domain — which stops your own team, but not an admin who can rebuild the vault. Pair with an external trust anchor and key ceremonies outside the operating team for evidence that holds against the insider scenario.
pipeline stage 4 — Output audit & human-in-the-loop gateway
Model Drift & Accuracy Monitor
Continuous evaluation against golden sets and sampled human verdicts; raises drift alerts and feeds the recertification cycle.
PII Scrubbing / DLP-NER Layer
Automated detection, pseudonymisation and blocking of personal data in inputs, retrievals and outputs.
pipeline stage 1 — Ingestion & data isolation
Cognitive Orchestrator
The reasoning and control plane of an agentic workflow: goal decomposition, tool selection across enterprise APIs, confidence scoring per step, and a human-machine interface exposing progress, limitations and a global halt. It is the architectural home of AI Act Art. 14 oversight — oversight that lives only in a downstream UI cannot stop an executing agent.
Dual-Gate Validation Pipeline
Input and output validation as two independent gates (MLCommons-hazard-class semantic filters, groundedness checks, structural validators: LLM Guard sub-ms–10 ms, Llama Guard <90 ms, NeMo <50 ms, Guardrails AI 50–200 ms). Latency economics decide the architecture: sequential gate chains add 300–800 ms per agent action; parallel evaluation collapses total added latency to the slowest single check — run independent checks concurrently, reserve sequential ordering for true dependencies.
Materiality-Threshold Escalation
Autonomy is bounded by pre-configured limits — variance thresholds, disbursement caps, margin floors, confidence minima. Crossing a limit halts execution and routes the case to a named human with the synthesised context, rather than letting the agent proceed at degraded confidence.
Shadow-Mode Execution
Run governance controls in observe-and-score mode before enforcement: the policy engine and guardrails evaluate every agent action and log verdicts without blocking, yielding empirical false-positive/negative rates and calibrated thresholds. De-risks the enforcement cutover, produces baseline evidence for Art. 9 risk estimation, and is the standard migration path when retrofitting controls onto a live workflow.
Build or Buy — Vendor Layer (4)
The graph models vendor CATEGORIES as first-class nodes and keeps named vendors as community-maintained, disputable desc content with lastVerified dates. A category is stable; a vendor list is a currency-layer object like any standard node.
AI GRC & Governance Platforms
Second-line systems of record: model/agent inventory incl. third-party SaaS AI, automated risk tiering, policy administration, cross-framework mapping & control deduplication, audit-evidence generation, intake workflows. Exemplary (community-maintained): ModelOp Center, Credo AI, IBM watsonx.governance, OneTrust, Holistic AI, Modulos (governance graph), Monitaur (insurance/lending), Fairly AI, Saidot, Trustible, Enzai, LatticeFlow (technical validation), Vanta (evidence automation), ServiceNow (intake/ITSM); data-catalog adjacency: Collibra, Alation, Informatica. Selection metrics: see meta.marketLandscape.selectionMetrics.grc.
unverified · verified 2026-08-06 community-maintained
selection metrics: multi-model/multi-cloud cataloging incl. third-party SaaS, automated risk tiering, regulatory reporting, independent-2nd-line deployability, cross-framework control deduplication
supplies: Live Risk Register / Posture Management · AI Register & Model Registry / Factsheets · AI Intake Portal & Use-Case Triage · Vendor & Model Due-Diligence Kit
Secure Data Infrastructure & Vector Storage
Governed retrieval substrate: vector databases, lakehouses and catalogs with tenant/namespace isolation, RBAC + client-managed keys (CMEK), lineage into RAG chunks, air-gap options. Exemplary (community-maintained): Pinecone (serverless, SOC 2), Chroma/FAISS (self-hosted/air-gapped sovereignty), Snowflake Cortex (masking, clean rooms), Databricks Unity Catalog (end-to-end lineage), Azure AI Search, AWS OpenSearch. The Art. 10 runtime data-governance duties land here.
unverified · verified 2026-08-06 community-maintained
selection metrics: namespace/tenant isolation, RBAC + CMEK, lineage into RAG chunks, SOC 2 / ISO 27001 attestations, air-gap capability
supplies: Data Lineage & Versioning · Retrieval Rails (ACL-aware RAG) · Sovereign Context Layer
Agent Orchestration & SDLC Toolkits
Developer middleware for multi-agent networks, tool-use chains, RAG abstraction, state/memory persistence and model routing. Exemplary (community-maintained): LangChain, LlamaIndex, AutoGen, CrewAI; MCP-based tool ecosystems. Regulatory posture: orchestration code is where autonomy tiering, propose-action objects and fallback routing get implemented — the framework choice constrains which controls are cheap and which are retrofits.
unverified · verified 2026-08-06 community-maintained
selection metrics: broad model-API abstraction, state/memory management, error recovery, fallback routing hooks
supplies: HITL Escalation Queue & Review UI · Cognitive Orchestrator · Materiality-Threshold Escalation
Runtime Security & Guardrail Vendors
First-line inline enforcement: single-pass parallel input/output evaluation proxies, injection & exfiltration defense, PII masking, grounding checks, SecOps routing. Exemplary (community-maintained): Prompt Security, HiddenLayer (MLSDR), Palo Alto AI Runtime Security, AWS Bedrock Guardrails, NVIDIA NeMo Guardrails, Guardrails AI, Robust Intelligence, LLM Guard / Llama Guard OSS class. Selection metrics: single-pass latency (<20 ms class), catch rates, policy-version telemetry into the AI-BOM.
unverified · verified 2026-08-06 community-maintained
selection metrics: single-pass parallel evaluation latency (<20 ms class), injection/hallucination catch rates, SecOps/SIEM routing, policy versioning surfaced into the AI-BOM
supplies: Input Rails / Prompt Shields · Output Rails / Groundedness Check · Dual-Gate Validation Pipeline · Guardrail Sidecar / Interception
Procurement rule: Derived from three-lines-of-defense separation: the second-line GRC platform must be procured and deployed independently of any first-line runtime or model vendor — a governance tool that only sees its own vendor's models cannot govern a multi-model estate, and closed third-party SaaS AI can only be governed contractually (intake, attestation, AI-BOM disclosure), never by inline inspection.
Outsourced delivery BPO · SaaS · Service-as-a-Software caveats
Delivery Model — BPO · SaaS · Service-as-a-Software
Spectrum
BPO: input-priced (billable hours/FTEs), linear headcount scaling, human error & attrition as primary risk
SaaS: capability-priced (software access), client operates the workload, implementation/adoption failure as primary risk
Service-as-a-Software: outcome-priced (SLA on completed work), provider-managed AI executes 60–80% of cognitive tasks with specialist supervision, algorithmic bias & non-compliance as primary risk
Caveats in regulated markets
— Outcome SLAs move compliance risk onto the provider — but NOT the buyer's deployer duties: Art. 26 oversight, log retention and FRIA obligations stay with the enterprise even when execution is outsourced.
— Provider role analysis is the central legal question: a productized platform that fine-tunes, re-purposes or chains models can flip into the Art. 25 provider role with full high-risk obligations.
— Certified operations (ISO 42001) function as a procurement moat and shortcut third-party risk assessment — but organizational certificate ≠ product conformity (never conflate, see meta.assuranceEcosystem).
— The buyer's evidence chain must reach into the provider: contractually mandated AI-BOM disclosure, ZDR certificates, bias-audit reports and logging-ledger access are the artifacts that make an outsourced workflow auditable.
Threat Profile
LLM09 Misinformation
Hallucinated or wrong outputs create liability and decision risk.
mitigate with: Output Rails / Groundedness Check, Explainability API (SHAP/LIME/CoT)
Screening False Positives / Negatives
Automated identity, sanctions and PEP matching either floods analysts with false alerts or silently misses a true hit — both are supervisory findings.
mitigate with: Confidence Scoring & Threshold Gate, HITL Escalation Queue & Review UI, Bias Testing & Data Quality Pipeline
Demographic Bias in Automated Screening
Name, language or geography features act as proxies for protected characteristics, producing systematically different outcomes across groups.
mitigate with: Bias Testing & Data Quality Pipeline, Algorithmic Bias & Fairness Audit Report
LLM06 Excessive Agency
Over-broad rights/functions of autonomous agents lead to uncontrolled actions.
mitigate with: MCP Gateway / Proxy, Agentic Zero Trust, Per-Action Autonomy Tiering, Trinity Defense (TCB + Command Gates + IFC), Deterministic Policy Engine (OPA / Cedar), Guardian Agents (Runtime Policy Enforcement)