Learn / Agent and agentic system security: tool use, permissions, memory, multi-agent
Agent and agentic system security: tool use, permissions, memory, multi-agent
Securing AI systems that plan, call tools, hold memory, and act semi-autonomously, often through multiple cooperating agents.
Researched on 2026-09-26 with AI assistance. Links and summaries can change; verify details with the original source. Not yet reviewed by a person.
What it is
Agent security covers systems that combine model reasoning with external tool calls, where data returned by a tool can hijack the agent into doing something else. The failure mode is not a bad answer, it is an unwanted action taken with the agent's credentials.48
The core reason agents are hard to secure is that a model cannot reliably tell operator instructions apart from instructions that arrive inside content it reads, because everything ends up as one token sequence, and the system is non-deterministic so a defence that works once may fail next time.8
OWASP treats agentic AI as its own threat-model problem rather than a variant of chatbot risk, and its Agentic Security Initiative published a threat-model-based reference of agentic threats and mitigations in February 2025 as the first of a series.1
The scope now splits into layers: the protocol layer that connects a model to tools, the skills or behaviour layer that decides what those tools actually do in a multi-step workflow, and the memory that carries state between sessions.79
Why postings ask for it
31 of 48 postings (65%) ask for agent and agentic system security, and it is the top-weighted skill for builders and designers: 81% of the 16 AI/Agent Security Engineer roles and 86% of the 7 AI Security Architect roles.P
Red team and research roles ask at 78% of 9 and 75% of 4, which matches work like proving an agent can be driven into a destructive tool call, for example using a debug SQL tool to delete a user record.P5
Demand is much lower in AI Governance / GRC (14% of 7) and Consulting (20% of 5), though NIST is drafting SP 800-53 control overlays with separate single-agent and multi-agent use cases, so the control-mapping side of this skill is arriving for those roles too.P6
Concepts you should be able to explain
If you can say each of these out loud in two minutes, with an example, you are ready for the technical part of an interview on this skill.
An agent becomes a data-theft tool when three things meet in one session: access to private data, exposure to attacker-controlled content, and a way to communicate outward. Any single leg is usually fine; the combination lets an attacker instruct the agent to fetch secrets and send them out. Almost any HTTP-capable tool, including image loading or a clickable link, counts as the exfiltration leg.8
The attacker does not talk to the agent directly; they plant instructions in a web page, an issue, an email or a document the agent will read. Vendors have repeatedly shipped and then fixed this class of bug, usually by closing the exfiltration path rather than by making the model obedient only to its operator. Telling the model in its own prompt to ignore injected text is not a dependable control.84
Agents are often wired to tools that are far broader than the task needs, such as a debug interface that accepts an arbitrary SQL string. Asking the agent to enumerate its own APIs and their arguments is usually enough to find the over-broad one, and then a normal-looking request turns into a destructive write. The fix is scoping the tool, not scolding the model.5
MCP proxy servers that front a third-party API with one static OAuth client ID, while letting MCP clients register dynamically, can be tricked into handing an authorization code to an attacker's redirect URI because the user's earlier consent cookie suppresses the consent screen. The protocol's own guidance requires per-client consent stored server side, exact redirect_uri matching, and CSRF protection before the third-party flow starts. This is classic delegated-authorization work, not model work.2
Agents built on frameworks that keep mutable state carry corruption forward, so a single poisoned memory write can cause misaligned behaviour, data exfiltration or malicious actions in later sessions. OWASP now runs a dedicated project on guarding persistent agent memory. Treat memory writes as a trust boundary with their own validation and retention rules.9
Between the model and the tools sits a layer of skills or manifests that define how multi-step workflows are orchestrated, not just which resources are reachable. OWASP's Agentic Skills Top 10 argues this layer has been under-protected relative to the model and the tool protocol, and maps each risk to the Cloud Security Alliance MAESTRO seven-layer model for agentic systems. The short mental model: the protocol is how the model talks to tools, skills are what those tools actually do.7
Research from ETH Zurich and collaborators proposes design patterns for building agents with provable resistance to prompt injection, by constraining the agent's structure rather than trying to detect malicious text. The patterns trade utility for security in different ways, which is the conversation to have with product teams. Expect to argue about what capability you remove, not which classifier you buy.3
AgentDojo evaluates agents that use tools over untrusted data across 97 realistic tasks such as email, e-banking and travel booking, with 629 security test cases, and is designed as an extensible environment rather than a fixed test suite. Its results show current models fail many tasks even with no attacker present, and published injection attacks break some security properties but not all. So report utility and attack success side by side, and re-run as attacks evolve.4
Use fictional data and authorised sandboxes. Remove employer details and secrets from any portfolio write-up. Time estimates exclude setup. Check model and cloud costs before running tests, set spending limits, and delete lab resources afterwards.
Three exercises
In order of difficulty. Free tools. Keep what you build; it is evidence.
You can demonstrate, with steps, how an agent with one broad tool leads to data loss, and state the scoping fix.5
- Open the PortSwigger excessive agency lab and use its live chat.
- Ask the agent which APIs it can call, then which arguments each takes.
- Note the debug interface that accepts a whole SQL statement.
- Read a record, then perform the destructive write that solves the lab.
- Write three sentences on which tool contract change would have blocked it.
Tools: PortSwigger Web Security Academy account, browser
You can produce a one-page review that says which tool combinations allow data theft and whether the authorization flow is a confused deputy.82
- List every tool your agent or IDE assistant can call, from its config files, recording tool and server names only and keeping tokens and keys out of the write-up.
- Tag each tool as private-data access, untrusted-content intake, or outbound communication.
- Mark every session where all three tags appear together and name the exfiltration path.
- For any proxy server, check per-client consent, exact redirect_uri matching and CSRF protection.
- Recommend one capability removal per trifecta rather than a prompt-level warning.
Tools: text editor, spreadsheet, your existing MCP client config
You can show a table of task completion and attack success for one agent configuration, and defend the numbers.43
- Install AgentDojo from the code released with the paper and run a task suite with no attack.
- Record baseline task completion so you know the agent's honest failure rate.
- Re-run with the bundled injection attacks and record attack success per security test case.
- Enable one defence paradigm, re-run, and note what utility you lost.
- Write up which security properties still broke and what you would test next.
Tools: Python, AgentDojo, a local model runtime or a free API tier
Practice questions
Written from the concepts above, not collected from a named employer. Open one, answer it out loud, then tick the points you covered; the score stays in this browser.
What is the lethal trifecta and how would you use it to triage an agent deployment?Lethal trifecta
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
Why is 'tell the model to ignore injected instructions' not an adequate defence?Indirect prompt injection through tool output
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
An agent has a debug tool that takes a raw SQL string. Walk me through the attack and the fix.Excessive agency
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
Describe a confused deputy problem in an MCP proxy server and the required protections.Confused deputy in tool protocols
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
How does persistent memory change the blast radius of a single successful injection?Memory poisoning and persistence
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
What is the difference between securing the tool protocol layer and the skills layer?The skills or behaviour layer
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
You are asked to prove an agent is 'safe against prompt injection'. What would you actually measure?Measuring agent robustness
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
A product team wants an agent with broad access and asks you for controls. How do you structure the answer?Design patterns over filters
Say your answer out loud or write it down, then tick what you covered:
0 of 4 covered
Sources
Every numbered claim above links here. P = the platform's own coding of 48 job postings.
- Agentic AI: Threats and Mitigations OWASP GenAI Security Project, Agentic Security Initiative
- MCP Security Best Practices Model Context Protocol
- Design Patterns for Securing LLM Agents against Prompt Injections arXiv
- AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents arXiv
- Lab: Exploiting LLM APIs with excessive agency PortSwigger Web Security Academy
- SP 800-53 Control Overlays for Securing AI Systems (COSAiS) NIST Computer Security Resource Center
- OWASP Agentic Skills Top 10 (AST10), version 1.0-2026 OWASP Foundation
- The lethal trifecta for AI agents: private data, untrusted content, and external communication Simon Willison
- OWASP Agent Memory Guard (project page, read from search result summary only) OWASP Foundation
Resources
Free first. Levels: intro means no prior knowledge of this skill; working means you can apply it on a project; advanced means research depth or specialist tooling.
- introAgentic AI: Threats and Mitigations OWASP GenAI Security Project (Agentic Security Initiative), Guide freeGives you the shared vocabulary for agent threats (tool misuse, memory poisoning, identity, cascading failures) before you design or test anything.EngineerRed teamerArchitectGovernanceConsultantResearcher
- introWeb LLM attacks (Web Security Academy topic) PortSwigger, Course freeTeaches how to map the tool and API surface an agent exposes, then exploit it, using methods pentesters already know from web testing.Red teamerEngineerConsultant
- introBack to the Future: Why Agentic AI Needs a Strong Identity Foundation NIST, Guide freeFrames agent permissions as an identity problem, giving architects and GRC leads language for delegation, attribution and non-human identity decisions.ArchitectGovernanceEngineerConsultant
- introLab: Exploiting LLM APIs with excessive agency PortSwigger Web Security Academy, Hands-on lab freeFirst hands-on proof that over-broad tool permissions, not the model, are what let an attacker delete a user.Red teamerEngineerConsultant
- introOWASP AI Exchange: AI Security Overview OWASP AI Exchange, Guide freeMaps AI threats to controls and a step-by-step risk analysis, so you can place agent risks inside an existing security programme.EngineerArchitectGovernanceConsultant
- introThe Agentic AI Security Scoping Matrix AWS Security Blog, Guide freeGives architects and GRC leads a way to classify how much autonomy an agent has and which controls that level of autonomy demands.ArchitectGovernanceConsultant
- workingLab: Exploiting vulnerabilities in LLM APIs PortSwigger Web Security Academy, Hands-on lab freeReaches OS command injection in the tool behind an agent, showing that classic appsec bugs in tools are the real blast radius.Red teamerEngineer
- workingMITRE ATLAS MITRE, Standard freeLets you write agent red team plans and detections against named tactics and techniques instead of ad hoc attack ideas.Red teamerEngineerArchitectResearcherGovernance
- workingOWASP Agentic Skills Top 10 OWASP, Standard freeCovers the skill and manifest supply chain (SKILL.md, skill.json, manifest.json) that agents load, a risk area the older agent lists miss.EngineerArchitectRed teamerGovernance
- workingMCP Security Best Practices (specification 2025-11-25) Model Context Protocol, Standard freeThe protocol authors' own list of attack vectors and required controls, which is what you check an MCP client or server implementation against.EngineerArchitectRed teamer
- workingSecuring Agentic Applications Guide 1.0 OWASP GenAI Security Project (Agentic Security Initiative), Guide freeTurns agent threat lists into build-time decisions on tool scoping, sandboxing, memory handling and human approval that you can put into a design review.EngineerArchitectConsultant
- workingPrompt injection (series) Simon Willison, Guide freeTracks why injection stays unsolved and which architectural patterns (dual LLM, limited privilege) practitioners actually adopt.EngineerRed teamerArchitectConsultant
- workingAgentic AI Threat Modeling Framework: MAESTRO Cloud Security Alliance, Guide freeGives architects a seven-layer model for locating agent threats (foundation model, memory, tools, orchestration, identity) during a threat modelling workshop.ArchitectGovernanceConsultantEngineer
- workingTechnical Blog: Strengthening AI Agent Hijacking Evaluations NIST, Guide freeShows why weak hijacking tests understate risk and how to strengthen attack sets, which is how you justify your own agent test results.Red teamerResearcherEngineer
- workingWeb LLM attacks learning path PortSwigger Web Security Academy, Course freeTeaches how to map an agent's tool and API attack surface, then exploit it, with graded labs and no setup.Red teamerEngineerConsultant
- workingLab: Exploiting AI agents to perform destructive actions PortSwigger Web Security Academy, Hands-on lab freeYou plant content that an autonomous scanning agent reads with borrowed credentials, the core indirect injection pattern in real agent incidents.Red teamerEngineerResearcher
- workingAgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents arXiv (Debenedetti et al.), Paper freeThe default benchmark for measuring whether an agent defence actually works; read it before you claim any mitigation is effective.ResearcherRed teamerEngineer
- workingSP 800-53 Control Overlays for Securing AI Systems (COSAiS) NIST, Standard freeFollow the drafts that will map agent and AI risks onto SP 800-53 controls, which is how your audit and ATO evidence will be framed.GovernanceArchitectConsultant
- workingAgent Control Standard (ACS) OWASP GenAI Security Project, Standard freeNew OWASP standard defining runtime hooks at tool call, memory write and sub-agent invocation, so engineers and architects can specify enforceable agent controls instead of prompts.
- workingATLAS data changelog: content releases 2026.07, 2026.08 and 2026.09 MITRE ATLAS, Standard freeATLAS now tags techniques by Agentic AI platform and renamed agent tool poisoning, supply chain and memory hardening entries, so red teams can remap existing agent test plans.
- workingAI Coding Agent Sandbox Escapes: The Trust Handoff Flaw Cloud Security Alliance, Guide freeDocuments how delayed-trust file writes bypassed sandboxing in Cursor, Codex, Gemini CLI and Antigravity, giving AppSec and pentest staff a reproducible class of agent escape to test.
- workinghoop: runtime enforcement proxy for AI agent access hoop.dev (GitHub), Tool freeActively maintained open-source proxy that masks sensitive data and blocks destructive commands from agents, giving cloud and platform engineers an enforcement point to trial.
- advancedDesign Patterns for Securing LLM Agents against Prompt Injections arXiv, Paper freeGives named architectural patterns that constrain what an agent can do, so you can review a design instead of only filtering prompts.ArchitectEngineerResearcherConsultant
- advancedSecurity Considerations for Model Context Protocol (MCP) Implementations in AI Agent Systems (IETF draft) IETF (Internet-Draft), Standard freeClassifies recurring MCP vulnerability classes from publicly reported cases, useful when writing your own MCP security requirements.ArchitectEngineerResearcherGovernance
- advancedThreat Modeling Google's A2A Protocol with the MAESTRO Framework Cloud Security Alliance, Guide freeWorked threat model of an agent-to-agent protocol, covering agent card spoofing, delegation and trust between agents you do not control.ArchitectResearcherConsultant
- advancedLab: Exploiting AI agents to exfiltrate sensitive information PortSwigger, Hands-on lab freeYou plant content that redirects a credentialed scanning agent into leaking API keys, the exact pattern behind most real agent data loss.Red teamerEngineerResearcher
- advancedDefeating Prompt Injections by Design (CaMeL) arXiv (Google, Google DeepMind, ETH Zurich), Paper freeShows how separating planning from data and enforcing capability-based data flow can stop injection without trusting the model, the strongest current defence idea.ResearcherArchitectEngineer
- advancedIndirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks? arXiv, Paper freeArgues a modular defence at the agent-tool interface clears four public benchmarks, and that this means the benchmarks are too weak.ResearcherEngineerRed teamer
- advancedAI Red Teaming, Prompt Hacking and AI Security Masterclass Learn Prompting, Course USD 1199Live instruction and feedback on red team methodology, the one thing self-serve labs and papers do not give you.Red teamerConsultant
Gaps the research could not fill with a good free source: No free, self-hosted vulnerable multi-agent range or agentic CTF found today: GitHub searches for vulnerable MCP or agent labs returned only scanners, so all hands-on items are hosted single-agent labs.; No second free structured course dedicated to agent security with stated hours; PortSwigger's learning path is the only free curriculum confirmed.; Costs, durations and prerequisites are unconfirmed across the index: the tool budget ran out before any provider page could be fetched.; No ENISA, UK NCSC or ISO document specific to agentic systems surfaced; the closest governance material is the in-progress NIST COSAiS overlays and NIST Cyber AI Profile workshop slides.; Nothing found on agent-to-agent protocol security (for example A2A) from a standards body, and no free source on identity and non-human credential management for fleets of agents.; No free detection and monitoring guidance for SOC teams: no source found on what agent trajectory telemetry to log or alert on.
Paid options
Most of what postings ask for on this skill is covered by the free material above. These are the paid courses and certifications that touch it, with what they add and what free already covers. Showing 6 of 8: ones postings name first, then the most focused on this skill. All paid options.
- Cost
- not stated on the provider page
- Duration
- 11 modules; self-paced, sold by access duration (lab extensions in 30-day increments for Course & Cert Bundle learners)
- Format
- self-paced
- Prerequisite
- As stated: advanced level, for experienced cybersecurity practitioners, red teamers and AI professionals; solid cybersecurity fundamentals and basic familiarity with AI systems including LLMs
- Renewal
- OSAI does not expire; the OSAI+ designation expires 3 years from issuance, maintained by one of three continuing education paths. Passing OSAI+ may qualify for 40 CPE points, self-submitted to ISC2
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: The first graded, proctored AI red team exam from a provider the two AI red team postings already name, against live LLM and agent-integrated targets.
Free already covers: OWASP LLM Top 10, prompt injection writeups and free CTF-style prompt hacking games cover most attack classes with no proctored assessment.
- Cost
- not stated on the provider page
- Duration
- 30+ guided exercises, 60 days browser-based lab access, 36 CPE points, one exam attempt included
- Format
- self-paced
- Prerequisite
- not stated
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: One of the few paid exams scoped to Model Context Protocol specifically: attacking, assessing and hardening MCP servers including tool poisoning, prompt injection, supply chain and agentic defences.
Free already covers: The MCP specification, OWASP agentic security guidance and public MCP tool-poisoning research already describe these attack patterns, and you can run a vulnerable MCP server locally for free.
- Cost
- not stated on the provider page
- Duration
- 11 modules, self-paced; sold via Course and Cert Exam Bundle, Learn One or Learn Enterprise from 31 March 2026; lab extensions in 30-day increments for bundle learners
- Format
- self-paced
- Prerequisite
- advanced level: solid cybersecurity fundamentals and basic familiarity with AI systems including LLMs; aimed at experienced practitioners, red teamers and AI professionals
- Renewal
- OSAI does not expire; the OSAI+ designation expires 3 years from issuance and is maintained through one of three continuing education paths
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: A proctored offensive exam against AI-integrated environments from the vendor whose other certs the red-team postings do ask for, worth 40 ISC2 CPE points on a pass.
Free already covers: Microsoft's free AI Red Teaming material and PyRIT, plus OWASP LLM guidance and public jailbreak research, teach the techniques; what you cannot get free is the graded exam.
- Cost
- not stated on the provider page
- Duration
- not stated on the page I fetched; the AIRTP+ exam is a 24-hour practical assessment and the on-demand version of the masterclass is listed as 25 hours
- Format
- mixed
- Prerequisite
- not stated
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: Practice in the HackAPrompt playground with instructors from Microsoft's AI Red Team and top bug bounty hunters, plus explicit training in writing up reproducible findings and impact, which is the part clients pay for.
Free already covers: Free HackAPrompt-style challenge playgrounds, the team's own published research and OWASP LLM guidance cover prompt injection and jailbreak technique at no cost.
- Cost
- HTB Academy subscription rates from 12 October 2026: Student $10/mo, Silver $30/mo, Gold $95/mo, Platinum $125/mo, Silver Annual $550/yr (EUR 469 / GBP 416), Gold Annual $1,400/yr (EUR 1,195 / GBP 1,060); individual modules can still be bought with cubes and this path requires 970 cubes
- Duration
- 12 modules, 230 sections; no hour estimate given
- Format
- self-paced
- Prerequisite
- none stated; path difficulty is listed as Hard and modules run Medium to Hard
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: Graded browser labs that take you from ML fundamentals through prompt injection, output handling abuse, data-pipeline poisoning and gradient-based adversarial attacks, aligned to Google's Secure AI Framework, with an exam-backed certificate at the end.
Free already covers: Free playgrounds such as prompt-injection challenge sites, OWASP LLM and Agentic guidance, and Microsoft's free AI Red Teaming 101 series cover the same attack list without labs that are graded.
- Cost
- not stated on the provider page
- Duration
- 7 hours self-paced, 7 CPEs
- Format
- self-paced
- Prerequisite
- at least intermediate Python; 16 GB RAM, 20 GB free disk, Rancher Desktop or Docker (Intel and ARM both supported); internet access needed during class
- In the 48 postings
- Not named in any of the 48 postings.
Adds over free material: Dockerised labs where you build RAG, contextual RAG and agentic RAG yourself and wire in access-control enforcement and prompt-injection defences, so you can review a retrieval pipeline you have actually built.
Free already covers: Vendor and framework docs plus OWASP LLM guidance already explain RAG architecture and the injection risks at concept level, and open tutorials show how to stand a RAG stack up.