Pidoku

Governance and Regulation

Advanced 55 min Difficulty 3/5 Lesson 06 of 06

Prerequisites Security, Privacy and Residency

The idea in one minute#

AI governance is how an organisation decides which AI systems may exist, under what conditions, and who answers for them. It has four working parts: an inventory of every AI system, a risk classification for each, controls that scale with the risk, and evidence that the controls are operating.

Regulation now sets a floor under this. The EU AI Act is the most detailed law so far: it sorts systems into risk classes and attaches obligations to each, with dates. Voluntary frameworks — the NIST AI Risk Management Framework and the certifiable standard ISO/IEC 42001 — give the structure most organisations use to meet it.

An architect is not the organisation’s lawyer. But the architect makes the decisions that determine which rules apply — what the system does, to whom, with how much autonomy — and produces most of the evidence. A design that knows its risk class from the first page passes governance in weeks. One that discovers it at launch does not launch.

Dates and statuses here were checked on 13 October 2026. This is orientation for engineers, not legal advice.

A picture#

flowchart LR
  UC[":i-file-text: <b>Proposed AI system</b>"] --> INV[":i-list-checks: <b>Register in the inventory</b><br/><small>owner, purpose, data, model</small>"]
  INV --> CL{"Risk class?"}
  CL -->|"prohibited"| NO[":i-ban: <b>Do not build</b>"]
  CL -->|"high"| HI[":i-shield-alert: <b>Full controls</b><br/><small>risk management, documentation,<br/>human oversight, logging, assessment</small>"]
  CL -->|"transparency"| TR[":i-eye: <b>Disclose</b><br/><small>tell people it is AI; mark generated content</small>"]
  CL -->|"minimal"| LO[":i-check: <b>Baseline controls</b><br/><small>security, privacy, evaluation</small>"]
  HI --> EV[(":i-archive: <b>Evidence</b><br/><small>design doc, evals, logs, reviews</small>")]
  TR --> EV
  LO --> EV
  EV --> RV[":i-recycle: <b>Review</b><br/><small>on change, and yearly</small>"]
  RV -.-> INV
  class UC neutral
  class INV,CL queue
  class NO,HI warn
  class TR,LO compute
  class EV memory
  class RV io

How it really works#

The four working parts#

PartWhat it isThe architect’s contribution
InventoryA register of every AI system: owner, purpose, users, data, model, vendor, autonomy level, risk classAn entry for each design, created at the start
Risk classificationA repeatable way to sort systems by potential harmThe facts that decide the class; an early, written classification
Controls by tierHigher risk, more required: review depth, oversight, testing, documentationA design that includes the controls for its tier
EvidenceRecords that controls exist and workThe design document, ADRs, evaluation results, logs, review findings

Governance fails in two opposite ways. Too little, and an incident produces a blanket ban. Too much — every chatbot through a six-month committee — and people route around it with personal accounts, which is the worst outcome of all. Tiering is what prevents both: a light path for low-risk systems makes the heavy path for high-risk ones credible.

The EU AI Act in one table#

The Act applies to organisations that place AI systems on the EU market or whose systems' output is used in the EU, wherever they are based. It classifies by use, not by technology.

ClassWhat falls in itWhat is required
ProhibitedPractices judged unacceptable — for example social scoring, manipulative techniques that cause harm, untargeted scraping to build face databases, emotion inference in workplaces and schoolsMust not be built or used
High-riskSystems used in listed areas (Annex III) — including employment and worker management, access to essential services such as credit scoring and risk pricing for life and health insurance, education, law enforcement, migration, critical infrastructure — and AI that is a safety component of regulated products (Annex I)A risk-management system, data governance, technical documentation, logging, transparency to users, human oversight, accuracy and robustness, conformity assessment, registration
Transparency obligationsSystems that interact with people, generate synthetic content, or do emotion recognition or biometric categorisationPeople must be told they are dealing with AI; generated content must be marked
Minimal riskEverything else — most internal assistants and productivity toolsNo specific obligations under the Act; other law still applies
General-purpose AI modelsThe foundation models themselvesObligations fall on the model provider: documentation, copyright policy, and more for models with systemic risk

Two roles matter to a designer. A provider develops a system and places it on the market or puts it into service under its own name. A deployer uses a system in its own operations. Most enterprises are deployers of someone else’s model — but building a system on top of that model and offering it, or substantially modifying one, can make you a provider of that system, with the heavier duties.

The dates#

The schedule was amended in July 2026 by the Digital Omnibus on AI — Regulation (EU) 2026/1744, published on 24 July 2026 and in force from 27 July 2026 — which postponed the high-risk obligations and left the rest in place.

2 Feb 2025    Prohibitions apply. AI literacy duty applies.
2 Aug 2025    Obligations for providers of general-purpose AI models apply.
2 Aug 2026    Transparency obligations (Article 50) apply.
2 Dec 2026    Two additional prohibitions from the Omnibus apply; reported end of the
              grace period for marking content from systems already on the market.
2 Dec 2027    High-risk obligations for stand-alone systems (Annex III).   ← was Aug 2026
2 Aug 2028    High-risk obligations for AI in regulated products (Annex I). ← was Aug 2027

What that means for a design started today: prohibitions and transparency duties already apply, so a customer-facing assistant must disclose that it is AI now. A high-risk system has until late 2027 — which, for a system with a multi-year life, means it should be designed to the high-risk requirements from the beginning. Deferred is not cancelled.

Classifying a use case#

Five questions, asked in discovery and recorded in the design document.

  1. Is the practice prohibited? If yes, stop.
  2. Is it in a listed high-risk area, or a safety component of a regulated product?
  3. Does the output materially influence a decision about a person — hiring, credit, insurance, access to a service?
  4. Does it interact with the public or generate content people might take as human-made?
  5. What is our role — provider or deployer?

The claims example from this course shows why wording matters. Extracting facts from claim documents for a handler who decides is assistance. Deciding automatically who is paid changes who is accountable and brings automated-decision rules into play, and assessing risk or setting prices for individuals in life or health insurance is a listed high-risk use. The technology is identical; the classification, the controls and the timeline are not. Scope is a governance decision.

NIST AI Risk Management Framework#

The NIST AI RMF is voluntary and widely used as the organising structure for an AI risk programme, inside and outside the United States. It has four functions.

FunctionMeaningWhat an architect supplies
GovernPolicies, roles and accountability existA named owner; the system’s entry in the inventory
MapThe context and the risks are understoodPurpose, users, affected people, data flow, risk list
MeasureRisks are assessed and trackedEvaluation results, red-team findings, monitoring
ManageRisks are prioritised and acted onControls, residual risk accepted by a named person, incident plan

NIST published a Generative AI Profile (AI 600-1) in July 2024 that applies the framework to generative systems. In February 2026 NIST’s Center for AI Standards and Innovation launched an AI Agent Standards Initiative, covering agent identity, security and interoperability; its outputs are still emerging and worth tracking if you design agents.

ISO/IEC 42001#

ISO/IEC 42001, published in 2023, is a management-system standard for AI — the same kind of standard as ISO 27001 for information security. An organisation can be audited and certified against it. It does not say how to build a system; it requires that the organisation has a working process for deciding, assessing, controlling and improving its use of AI.

Its value to an architect is practical: customers and regulators increasingly ask for it, and an organisation that holds it already has the inventory, risk assessment and review cycle your design needs to plug into.

How the three fit together#

EU AI Act        the law      what you must do, for which systems, by when
NIST AI RMF      the method   how to organise the work of identifying and managing risk
ISO/IEC 42001    the proof    an audited management system that shows you do it

Sector rules sit on top — financial services, healthcare, the public sector — and privacy law applies throughout. A growing list of other jurisdictions have their own AI rules; the approach of inventory, classification, controls and evidence satisfies most of them, which is why it is worth doing once, properly.

Governance by tier#

A workable internal scheme has three or four tiers. An example:

TierExamplesApprovalRequired before launch
1 — LowInternal drafting or search over non-sensitive content; a person uses the outputTeam lead; registered in inventorySecurity baseline; basic evaluation
2 — MediumInternal tools on sensitive data; customer-facing assistance with a person in the loopArchitecture and security reviewEvaluation gate; data-protection assessment; disclosure where public; monitoring
3 — HighInfluences decisions about people; acts autonomously on important systems; high-risk under lawAI governance boardAll of tier 2, plus formal risk assessment, documented human oversight, bias testing, adversarial testing, audit logging, legal sign-off
ProhibitedListed prohibited practices—Not built

The architect’s job is to propose the tier with reasons, design to it, and resist both temptations: under-classifying to move faster, and over-classifying to feel safe.

The evidence pack#

For a tier 2 or 3 system, assemble these as you go. Nearly all of them are artefacts you are already producing.

EvidenceComes from
Purpose, scope and non-goalsThe design document
Risk classification and reasoningThe design document
Data sources, flows, residency, retentionSecurity review
Model and vendor, with termsADR on model choice
Evaluation results, including slices and adversarial casesEvaluation gate
Human oversight designRequirements page
Known limitations and intended useWritten for users — a short “system card”
Monitoring and incident planOperations runbooks
Review record and residual-risk acceptanceWell-architected review; sign-off

AI literacy#

One obligation of the EU AI Act has applied since February 2025 and is easy to overlook: organisations must ensure that staff who operate or use AI systems have a sufficient level of AI literacy. For an architect this is an input to the rollout plan — the training line in the budget is a compliance item as well as an adoption one.

Remember this#

  • Governance is four parts: inventory, risk classification, controls by tier, evidence.
  • The EU AI Act classifies by use: prohibited, high-risk, transparency, minimal — plus duties on general-purpose model providers.
  • Prohibitions and transparency duties already apply. High-risk obligations begin 2 December 2027 and 2 August 2028 after the 2026 deferral.
  • NIST AI RMF is the method, ISO/IEC 42001 the certifiable proof, the AI Act the law.
  • How you scope the system decides its risk class. Classify on page one of the design.

Try it#

  1. Take three AI use cases from your organisation and answer the five classification questions for each. Did any land in a class you did not expect?
  2. Write the inventory entry for one of them: owner, purpose, users, data, model, vendor, autonomy level, risk tier.
  3. List the evidence pack for that system. Which items exist today, and which would have to be created?

Check yourself#

  1. What are the four working parts of AI governance?
  2. Why can the same technology be minimal-risk in one design and high-risk in another?
  3. Which EU AI Act obligations already apply, and which were deferred in 2026?
  4. What is the difference in purpose between NIST AI RMF and ISO/IEC 42001?

Sources#

↑↓ navigate↵ openesc close

drag to pan · scroll to zoom