What is a practical learning path for agentic AI systems and AI cybersecurity?

Start with the right mental model

An LLM application predicts text or structured output. An agent adds a loop: it observes a task and context, selects a tool or response, observes the result, and may continue. A reliable production system constrains that loop with schemas, policy checks, time and cost limits, approval gates, and a way to stop it.

This distinction matters because an LLM's output is untrusted input to the rest of the system. A persuasive completion is not proof that a user is authorized, that a retrieved document is true, or that a requested action is safe. Treat model output in the same spirit as a browser request or a form submission: parse it, validate it, authorize it, and log the meaningful security event.

Learn how applications retrieve information, call tools, and use integrations, alongside the framework you choose. Model Context Protocol (MCP) is one example of the integration layer. Its current specification says tools can represent arbitrary code execution, calls for explicit user consent, and its authorization specification requires resource-specific token validation and forbids token passthrough. MCP 2026-07-28 specification MCP 2026-07-28 authorization specification

A staged learning path

Use the stages in sequence, but keep a small project alive from stage 3 onward. Each stage should end in a public, reproducible artifact: source code, threat model, tests, a short architecture note, and a short demonstration using only synthetic data.

Stages 1 and 2: become hard to surprise

Build normal applications before agent workflows. Write a small API that stores a user's notes and exposes two roles: reader and editor. Test every endpoint with the wrong user, the wrong tenant, and no authentication. Learn why a token is evidence of an identity and scope, not a permission to do whatever a model suggests. Work through HTTP, access control, OAuth, server-side request forgery, injection, and API-testing labs in the PortSwigger Web Security Academy. Use only the lab platform and systems you own or are explicitly authorized to test.

Practice locally with OWASP WebGoat only in its intended isolated environment. WebGoat is deliberately insecure and OWASP warns that it makes the host vulnerable, so do not expose it to the Internet. The point is to recognize security boundaries and verify controls, not to scan unrelated websites. OWASP WebGoat safety guidance

Learn basic networking as an application developer: where a DNS name resolves, how HTTPS protects a connection but not authorization, how a reverse proxy changes headers, and how a service reaches a database. Learn identity as a security engineer: who requested an action, which tenant they belong to, which exact operation is permitted, and how that decision will be audited. These skills are more important for safe agents than memorizing a prompt framework.

Stage 3: understand enough ML to build useful LLM applications

You do not need to train a foundation model to become an agent engineer. You do need to understand data splits, overfitting, precision and recall, embeddings, similarity search, model-version changes, and the difference between a plausible answer and a verified answer. Build a retrieval-augmented generation system over a small public corpus such as your own project documentation. Store source identifiers with chunks and show them with every answer. Keep the corpus small enough to inspect manually.

Evaluate retrieval separately from generation. Create a held-out set of 20 to 40 questions with known supporting documents. Measure whether the correct source appears in the retrieved set before judging the final prose. Then label final answers as supported, unsupported, incomplete, or refusal. This makes failures diagnosable: poor retrieval needs data or search changes, while a response that ignores a retrieved source needs application or model changes.

Use structured output for anything downstream code will consume. For example, have the model propose {action, resource_id, rationale} and reject output that does not meet the schema. The schema only makes parsing reliable. It does not authorize the action. Your application must look up the current user, resource, and allowed operation independently before it invokes a tool.

Stage 4: add tools, then orchestration and observability

Start with one read-only tool such as search_public_runbook(query) rather than an email, shell, cloud-administration, or payment tool. Define the tool's input schema, maximum result size, time limit, and allowed data classification. Make the application return a clear "cannot perform that action" response when an action is outside the tool's contract. Only after this is robust should you add a sandboxed write tool, such as creating a ticket in a local mock service.

An agent framework can help represent state and calls, but it cannot choose your security policy. Prefer an explicit state machine for high-impact workflows: collect request, retrieve limited context, propose action, authorize, require approval if applicable, execute, verify result, and record outcome. Make destructive actions idempotent where possible, add request identifiers, cap loop iterations, cap spending, and provide a stop switch.

Observability means being able to reconstruct what the system did without storing sensitive prompt content indiscriminately. Capture a correlation ID, authenticated principal and tenant, model and application version, retrieval source IDs, policy decision, requested tool and validated parameters, approval outcome, result code, latency, cost, and error class. Redact secrets and sensitive customer content. OWASP notes that logs themselves can expose personal data, access tokens, connection strings, and keys, so restrict access and decide deliberately what is safe to retain. OWASP Logging Cheat Sheet

Evaluate the whole system continuously. Keep a versioned test set with ordinary tasks, ambiguous requests, malformed tool inputs, inaccessible documents, adversarial documents, and approval-denied cases. Track task success, grounded-answer rate, unauthorized action attempts blocked, sensitive-data leakage rate, false refusals, tool-error rate, latency, and cost. Review both aggregate numbers and a sampled trace. A high task-success score that silently performs an action under the wrong identity is a failure.

A security architecture worth practicing

Use a seven-stage boundary: user and untrusted documents, agent application, structured model proposal, schema validation and policy evaluation, identity and scoped authorization, approved tool gateway, then a sandboxed or protected downstream service. Send redacted audit events from the application, policy layer, and gateway to a separate monitoring path.

Do not treat one policy engine as a complete solution. Authorization belongs at every protected service, and the tool gateway should be able to deny a request even if the agent application fails.

Threat model before adding a capability

For every tool, write down the following before implementation:

1. Asset: What could be harmed or disclosed? Include records, credentials, money, external reputation, tenant boundaries, and production availability.
2. Actor and entry point: Who might influence it? Include a malicious user, a compromised document, a poisoned retrieved page, a package maintainer, and a confused legitimate user.
3. Trust boundary: Where does data cross from user, web page, email, attachment, memory, retrieval store, model, agent process, tool, and downstream service?
4. Abuse case: What unsafe behavior would success look like? Phrase it as a testable outcome, such as "the agent creates a ticket in another tenant" or "the agent sends a document excerpt to an unapproved domain."
5. Controls and evidence: Which server-side checks prevent it, and which automated test and audit event prove the check ran?

Use MITRE ATLAS to broaden the abuse-case list. ATLAS is a living knowledge base drawn from observed and realistically demonstrated AI attacks, so it is a better starting point for adversarial thinking than a generic list of model tricks. Map the threats to the NIST Cybersecurity Framework 2.0 functions: Govern, Identify, Protect, Detect, Respond, and Recover. CSF is not a product checklist, but it gives a common language for managing the surrounding cyber risk.

The main AI-specific risks and practical controls

Prompt injection. An attacker puts instructions in a user message, web page, email, document, or retrieved chunk to make the system disregard its intended task. Indirect prompt injection is especially relevant to retrieval and browsing agents because the adversary does not need to be the chat user. Treat every external document as untrusted data, label its source, minimize what reaches the model, and never let natural-language content directly select permissions or endpoints. Test with hostile documents that ask the agent to reveal its instructions, contact an outside address, or make an unrelated tool call. OWASP's current LLM application risk guidance is clear that no prompt-only defense is complete.

Data exfiltration. The agent may combine a sensitive retrieved record with a network-capable tool or an overbroad response. Apply access control before retrieval, retrieve only the current user's permitted documents, minimize and classify context, prevent arbitrary URLs or recipients, and inspect outbound requests at a gateway. Add a test record containing an obvious synthetic canary value. A passing test proves the canary is neither returned to an unauthorized user nor sent to a tool endpoint. Do not place real secrets in a red-team corpus.

Confused deputy and tool authorization. A powerful service can become a deputy that an attacker persuades into using its privileges on the attacker's behalf. Never use one broad service token for every user action. Carry the user's identity and narrow, operation-specific scope to the authorization decision; issue a separate, audience-bound downstream token when needed; and make the downstream service enforce its own tenant and action checks. The MCP authorization specification explicitly requires audience validation and says a server must not pass its inbound token to an upstream API, a useful concrete lesson for any tool protocol. MCP authorization and token handling

Excessive agency. Do not give an agent more tools, permissions, or autonomy than the task needs. Separate read and write tools, use least privilege, require a human confirmation for irreversible or high-impact changes, and show the user the exact target and intended effect. OWASP's Top 10 for Agentic Applications 2026 provides a current framework for risks created when agents receive tools, privileges, and autonomy.

Secrets and sensitive configuration. Keep keys in a proper secret-management system, issue short-lived credentials where possible, scope them to a service and operation, rotate and revoke them, and prevent them from appearing in prompts, traces, test fixtures, build logs, or source control. The OWASP Secrets Management Cheat Sheet recommends centralized provisioning, auditing, rotation, and lifecycle controls. A secret in a prompt or chat transcript should be treated as potentially exposed and rotated, not merely deleted from a visible message.

Sandboxing and resource abuse. If an agent can run code, parse files, browse, or access a command line, run that work in an isolated, disposable environment with a non-privileged identity, a read-only filesystem where possible, a per-run workspace, strict CPU, memory, process, time, and token budgets, and outbound network allowlists. Do not mount host credentials or the Docker socket. A sandbox limits blast radius; it is not a reason to permit arbitrary commands. Test that a job cannot read a neighboring workspace, reach a non-allowlisted address, exceed its quota, or retain data after cleanup.

Supply-chain risk. The attack surface includes model and embedding dependencies, container bases, prompt templates, data connectors, MCP servers, packages, CI workflows, and model artifacts. Pin versions and image digests, review new tools before enablement, lock dependencies, generate an inventory or SBOM, scan it, and verify provenance where available. SLSA defines provenance as verifiable information about where, when, and how an artifact was produced. SLSA provenance OpenSSF Scorecard OSV-Scanner

Logging, red teaming, and incident response. Red-team only systems you own or are authorized to assess. Begin with a fixed safe corpus of benign and adversarial prompts, files, and tool responses, then add manual review of high-impact flows. The open-source garak scanner can help exercise LLM-oriented assessment cases, but it does not replace a threat model or human review. When a test finds unsafe behavior, add the minimized case to regression tests. For a real incident, stop or constrain the affected tool, revoke relevant credentials and sessions, preserve redacted traces and versions, assess which data and actions were affected, notify the appropriate response owners, remediate the boundary failure, then rerun the regression suite before re-enabling the capability. NIST SP 800-61 Rev. 3 integrates incident-response recommendations with CSF 2.0.

Three portfolio projects that demonstrate real skill

All three projects should use synthetic or public data and a local mock service. Put the threat model, architecture, setup instructions, tests, screenshots of redacted traces, and a short retrospective in the repository. Never publish credentials, customer prompts, or attack instructions that could be misused against a real system.

Project 1: cited, read-only policy assistant

Setup. Build a RAG assistant over 30 to 100 public policy pages or fictional employee-handbook documents. It may search, quote short supporting passages, and link sources. It has no external action tools. Each document carries an owner, classification, and tenant label. Add a separate collection of malicious test documents that contain irrelevant instructions aimed at redirecting the assistant.

Threat model. An untrusted document author wants the assistant to disclose a restricted fictional document, claim unsupported policy, or follow text embedded in a retrieved page. Assets are tenant-scoped documents and answer integrity. Trust boundaries are document ingestion, vector retrieval, model context, and final response.

Tests. Test cross-tenant retrieval returns nothing; test a malicious chunk does not change the requested task; test every answer either cites a retrieved authorized source or abstains; test the synthetic canary never appears for an unauthorized user. Log source IDs and authorization decisions, not raw secret-like content. The tests give you evidence to discuss how retrieval, access controls, and prompt injection interact.

Project 2: approval-gated change-request agent

Setup. Create a local mock change-management API with two tools: get_change(change_id) and create_change_draft(service, summary, risk). The agent can read and draft only. A separate human reviewer approves a draft before a different component can apply any change in a fake environment. Use per-user roles and a tenant ID in every API request.

Threat model. A malicious requester tries to create a cross-tenant change, persuade the agent to skip review, or exploit a broad service credential. Assets are tenant isolation, the integrity of change records, and the fake environment's availability. Trust boundaries are user request, model output, policy gate, authenticated API call, and reviewer approval.

Tests. Test that the model's claimed role has no effect on authorization; only the authenticated identity does. Test a reader cannot draft, an editor cannot target another tenant, a request to apply a change is refused, and a compromised instruction in an attached ticket cannot override the approval gate. Verify that an action requires a fresh scoped token, an approval record, and an idempotency key. The result demonstrates least privilege, separation of duties, and a defense against confused-deputy behavior.

Project 3: sandboxed security-operations triage simulator

Setup. Feed an agent synthetic authentication, endpoint, and firewall events. Give it read-only search and a tool that creates a local incident draft. Run any parsing or analysis job in a disposable sandbox with no production credentials and an outbound network deny-by-default policy. The agent should classify confidence, summarize evidence, recommend a human next step, and attach relevant event IDs.

Threat model. An attacker controls a log field or attachment and attempts prompt injection, resource exhaustion, data leakage through a tool argument, or execution outside the workspace. Assets are synthetic but represent the confidentiality of telemetry, the integrity of incident records, and runner availability. Trust boundaries are telemetry ingestion, the model, tool calls, sandbox, logs, and incident API.

Tests. Include oversized input, malformed JSON, repeated tool-call suggestions, instruction-like text in a log field, a canary event from another tenant, and a request to run a command. Assert maximum calls and time are enforced, prohibited network access fails, cross-tenant data is absent, the command request is rejected, and logs are redacted. Add a tabletop incident drill: simulate a leaked test token, revoke it, prove later calls fail, and record the response timeline. The result demonstrates observability, resource controls, red teaming, and incident response without touching live infrastructure.

A twelve-week example plan

Assume eight to twelve focused hours per week. If you have less time, extend the calendar rather than skip tests and documentation. Keep a weekly engineering log with what failed, what you changed, and one risk you discovered.

This plan deliberately puts basic application security before unrestricted agent actions. The project may feel less flashy in week 5 than a multi-agent demo, but by week 12 it will be much more credible to an engineering or security reviewer.

How to choose courses, frameworks, and certifications

Choose courses by their evidence

Use this checklist before paying for or committing substantial time to a course:

• Does it state prerequisites and use a dated syllabus that can be checked against official documentation?
• Does it require you to build, test, and explain a project rather than only watch demonstrations?
• Does it teach retrieval evaluation, tool schemas, authorization, observability, failure handling, and cost limits?
• Does it include current AI security material such as prompt injection, data exposure, tool permissions, and supply-chain risk, with references to OWASP, NIST, or equivalent primary material?
• Does it distinguish a model's output from a policy decision made by trusted code?
• Does it offer a reproducible lab, source code, and a way to inspect tradeoffs, not just a vendor-specific visual workflow?

The Hugging Face course is a sensible no-cost starting point because it includes agent concepts, multiple frameworks, a final project, and modules on agentic RAG and observability. Pair it with the PortSwigger labs for application-security fundamentals and the NIST, OWASP, MITRE, and MCP documents for the security model. Do not try to complete every framework tutorial. Learn one framework well enough to compare it with an explicit state-machine implementation.

Treat certifications as supplementary

There is no universal certification that proves someone can secure production agentic systems. A foundational security or networking certification can help if target employers list it, if you need its structured syllabus, or if you lack a recognized baseline. A cloud or application-security certification is more useful when it matches the platform and role you are pursuing. It should come after enough hands-on work to understand why a control exists.

Before enrolling, read the current exam objectives on the issuer's site. Check that the exam is live, that it covers the job's actual areas such as identity, cloud, networking, application security, or incident response, and that you can afford renewal obligations. Treat a certificate as evidence of a bounded curriculum, not an assurance of ability. Your three project repositories should answer the more valuable questions: what did you secure, what did you test, which failures did you find, and how did you change the design?

Keep up without chasing hype

Set a monthly, time-boxed review: read updates from the OWASP GenAI Security Project, NIST AI RMF resources, MITRE ATLAS, the relevant protocol specifications, and one framework's release notes. Add a new threat or behavior to your test corpus only when it changes a real assumption in your system. That practice is more sustainable than rebuilding your projects for every new agent library.

For professional use, pair each new capability with a review question: "Which new data, identity, tool, or execution boundary did this introduce?" If the answer is unclear, leave the capability disabled. That is a sound response to an evolving field, not a sign that you are behind it.

Recommended starting sequence

If you are beginning today, complete the first two weeks of the plan, then build Project 1 before selecting an agent framework. Read the OWASP prompt-injection and excessive-agency pages while you create its hostile-document tests. Next, build Project 2 to practice authorization and approval design. Build Project 3 only after you can explain every permission and every log field in Project 2.

At every milestone, ask a reviewer to try to answer four questions from your repository: What can this agent do? What data can it see? Who authorizes each action? How would the team know and respond if it misbehaved? If your documentation and tests make those answers clear, you have a portfolio that aligns with the important direction of agentic AI and cybersecurity.