Short definitions of the terms this course uses. The lesson in brackets is where each is explained.
| Term | Meaning |
|---|---|
| Action rail | The deterministic check on every tool call: schema, authorisation, arguments, taint, approval. (Runtime Guardrails) |
| Adaptive attack | An attack tuned against a specific defence. (Prompt Injection) |
| AI-BOM | A bill of materials for a model: files, lineage, data, licences, evaluations. (Models and Supply Chain) |
| ASI01–ASI10 | The OWASP Top 10 for Agentic Applications. (Frameworks and Maps, Agent Threats) |
| ATLAS | MITRE’s matrix of adversary tactics and techniques against AI systems. (Frameworks and Maps) |
| Attack success rate | The share of attack attempts that achieve their outcome. (Build and Test) |
| Attestation | Hardware-signed proof of what software is running in a protected environment. (Deployment and Infrastructure) |
| Audience | The single resource server a token is valid for. (Identity and Authorization) |
| Backdoor | Behaviour planted in a model that appears only on a trigger. (Poisoning and Supply Chain) |
| CaMeL | A design that tracks the origin of every value and enforces policies on tool arguments. (Architectural Defenses) |
| Canary | A planted marker whose appearance in output or outbound traffic reveals a leak. (Data Exfiltration and Leakage) |
| Confidential computing | Hardware protection of data while in use, with attestation. (Deployment and Infrastructure) |
| Confused deputy | A program tricked into using its authority for someone who lacks it. (Why AI Security Is Different) |
| Denial of wallet | Driving up token or GPU spend to exhaust a budget. (Abuse and Resource Attacks) |
| Deterministic control | A control enforced by code or configuration, independent of model behaviour. (Threat Modeling an AI System) |
| Dual LLM | A privileged model that plans and a quarantined model that reads untrusted text. (Architectural Defenses) |
| Egress control | Restricting outbound network destinations. (Sandboxing and Egress) |
| Enterprise-Managed Authorization | The MCP extension by which an organisation’s identity provider governs access to servers. (Identity and Authorization) |
| Excessive agency | More functionality, permission or autonomy than the task needs. (Agent Threats) |
| Exfiltration | Moving data to where an attacker can read it. (Data Exfiltration and Leakage) |
| Guardrail | A runtime check on input, action or output. (Runtime Guardrails) |
| Disclosure of system prompts or other context the user should not see. (Frameworks and Maps) | |
| Indirect prompt injection | Injection delivered through content the system reads. (Prompt Injection) |
| Influence flow | Which text can steer which model, and what that model can do. (Threat Modeling an AI System) |
| Jailbreak | Getting a model to violate its safety training or operator rules. (Abuse and Resource Attacks) |
| Lethal trifecta | Private data, untrusted content and an outbound channel in one context. (Threat Modeling an AI System) |
| Memory poisoning | Planting instructions in an agent’s persistent memory. (Agent Threats) |
| OMS | OpenSSF Model Signing: a signature over all files of a model. (Models and Supply Chain) |
| On-behalf-of | An agent acting with a user’s delegated, scoped authority. (Identity and Authorization) |
| Probabilistic control | A control that estimates — a classifier, a judge model, an instruction. (Threat Modeling an AI System) |
| Prompt injection | Text that makes a model follow someone else’s instructions. (Prompt Injection) |
| Provenance | The recorded origin of data, a model, or an item in a context. (Data and Training, Monitoring and Response) |
| Quarantined model | A tool-less model used to read untrusted content. (Architectural Defenses) |
| Red teaming | Attacking your own system on purpose to find weaknesses. (The Red-Team Programme) |
| Rug pull | A tool or server that changes behaviour or descriptions after approval. (Poisoning and Supply Chain) |
| Rule of two | An agent session holds at most two of: untrusted input, sensitive access, external effect. (Architectural Defenses) |
| safetensors | A weights-only model file format that cannot execute code on load. (Models and Supply Chain) |
| Sandbox | An isolated environment for untrusted code. (Sandboxing and Egress) |
| Session taint | A sticky flag recording that untrusted content entered a context. (Architectural Defenses) |
| Token exchange | Trading one token for a narrower one naming subject, actor, audience and scope. (Identity and Authorization) |
| Tool gateway | The proxy between agents and MCP servers: catalogue, authorisation, inspection, audit. (MCP and Tool Security) |
| Tool poisoning | Instructions hidden in a tool’s description. (Poisoning and Supply Chain) |
| Trust boundary | A line in a design where control changes hands. (Threat Modeling an AI System) |
| Workload identity | A cryptographic identity for a piece of software rather than a shared secret. (Deployment and Infrastructure) |