Securing AI Features in AI Security & Guardrails

Securing AI Features is a chapter of the free AI Security & Guardrails tutorial with 7 lessons: Threat Modelling, Direct Injection & Jailbreaks, Indirect Injection, Sensitive Data & PII, Improper Output Handling, Excessive Agency and Guardrail Pipeline. Plan on about 270 minutes. It has 24 exercises you run and check in your browser.

What each lesson covers:

  • Threat Modelling: Every token the model reads is a possible instruction. Draw the trust boundaries of an AI feature, spot the lethal trifecta, and map each risk to the OWASP Top 10 for LLM apps.
  • Direct Injection & Jailbreaks: One customer message prints the system prompt and its secrets. See why prompt rules and classifiers leak, then stop every variant with architecture, policy in code and an output check, without blocking real customers.
  • Indirect Injection: A poisoned help-centre article makes the agent email customer data out, and the customer never sees it. Label untrusted text, take capabilities away and add an egress allow-list until no variant gets data out.
  • Sensitive Data & PII: The agent shows one customer another customer's order and writes it to the logs. Move scoping out of the prompt and into the tool layer, filter other people's details from replies, and redact the logs.
  • Improper Output Handling: A review runs a script in the next reader's browser, and a report request leaks every customer's rows. Encode what the chat renders and replace model-written SQL with a validated filter and parameters.
  • Excessive Agency: The agent pays a $420 refund nobody approved and gives a 100% discount on request. Remove the tools the job does not need, scope the rest and gate refunds in code, while valid refunds still go through.
  • Guardrail Pipeline: Put every control from Lessons 2 to 6 into one pipeline, measure it on mixed traffic with two numbers, choose a classifier threshold from its curve, and export your v1 security pack with an attack suite for Vitest.

When you finish, continue with Agents and Production.

Lessons in this section

  1. Threat Modelling

    Every token the model reads is a possible instruction. Draw the trust boundaries of an AI feature, spot the lethal trifecta, and map each risk to the OWASP Top 10 for LLM apps.

    • Explain why every token the model reads is a possible instruction
    • Mark the trust boundaries of an AI feature on its diagram
    • Spot the lethal trifecta: private data, untrusted content and a way to send data out
    • Map AI risks to the OWASP Top 10 for LLM Applications 2025
  2. Direct Injection & Jailbreaks

    One customer message prints the system prompt and its secrets. See why prompt rules and classifiers leak, then stop every variant with architecture, policy in code and an output check, without blocking real customers.

    • Explain direct prompt injection, jailbreaks and system prompt leakage
    • Show why a prompt rule only stops the phrasing it names
    • Measure a classifier's precision, recall and false positives
    • Move secrets out of the prompt and enforce policy in code
  3. Indirect Injection

    A poisoned help-centre article makes the agent email customer data out, and the customer never sees it. Label untrusted text, take capabilities away and add an egress allow-list until no variant gets data out.

    • Explain indirect prompt injection and why the victim never sees it
    • Show why labelling retrieved text helps but is not a wall
    • Remove a capability by splitting a reader agent from a privileged one
    • Build an egress allow-list for email, links and images
  4. Sensitive Data & PII

    The agent shows one customer another customer's order and writes it to the logs. Move scoping out of the prompt and into the tool layer, filter other people's details from replies, and redact the logs.

    • Explain why the model cannot keep customers' data apart
    • Scope every tool to the signed-in user in code
    • Show why a prompt rule and a reply filter each leave a gap
    • Redact other people's personal data in replies, keeping the customer's own
  5. Improper Output Handling

    A review runs a script in the next reader's browser, and a report request leaks every customer's rows. Encode what the chat renders and replace model-written SQL with a validated filter and parameters.

    • Treat model output as untrusted input to whatever uses it
    • Explain how a stored script reaches another customer's browser
    • Render replies safely while keeping Markdown formatting
    • Show why keyword deny-lists both miss attacks and block real users
  6. Excessive Agency

    The agent pays a $420 refund nobody approved and gives a 100% discount on request. Remove the tools the job does not need, scope the rest and gate refunds in code, while valid refunds still go through.

    • Name the three parts of excessive agency: tools, permissions and autonomy
    • Choose the smallest tool set for an agent's job
    • Scope tools that change data to the signed-in user
    • Gate refunds in code, including repeated refunds
  7. Guardrail Pipeline

    Put every control from Lessons 2 to 6 into one pipeline, measure it on mixed traffic with two numbers, choose a classifier threshold from its curve, and export your v1 security pack with an attack suite for Vitest.

    • Name the five stages of a guardrail pipeline and what belongs in each
    • Measure a pipeline on mixed traffic with block rate and false-positive rate
    • Read a classifier's threshold curve
    • Choose and justify a threshold against a false-positive budget