Curriculum › Security and Safety · 8.1% of the exam

Guardrails and safe deployment

What you'll be able to do

  • Layer guardrails rather than relying on any single one
  • Apply least privilege to agent credentials specifically
  • Choose secure-by-design defaults over opt-in hardening
  • Recognise when internal-only reasoning is being used to excuse a gap

What you’ll be able to do

  • Layer guardrails rather than relying on any single one
  • Apply least privilege to agent credentials specifically
  • Choose secure-by-design defaults over opt-in hardening
  • Recognise when internal-only reasoning is being used to excuse a gap

What you need to know

Layers, because none of them is sufficient alone

The named layers, each catching what the others miss:

  • System prompt hardening — shapes behaviour. Necessary, never sufficient.
  • Input filtering — catches known adversarial patterns before they reach the model.
  • Output filtering — catches leaked PII or policy violations before they reach the user.
  • Permission boundary — limits what a compromised or manipulated agent can actually do.

Exam answers offering exactly one of these as a complete solution are wrong on that basis. The right answer usually combines a detective layer with a preventive one.

Least privilege for agents

This deserves its own treatment because agents are launched by people, and inheriting the launcher's credentials is the path of least resistance.

An agent running with an engineer's full API access — including production database writes it never needs — is a finding on its own, before anything has gone wrong. The reasoning is simple: a manipulated agent can only exercise access it actually holds. Scoping credentials to the task bounds the blast radius of every other failure in the system.

ScenarioA team objects that their agent is internal-only, so the wide credentials don't matter.

Internal-only is not a control. It describes who can reach the agent, not what the agent does with content it processes — and indirect injection arrives through content, not through the network edge. An internal agent reading an uploaded document is exactly as reachable as an external one.

Secure by default

Defaults decide real-world posture, because most deployments never change them. A system where tools are permissive unless restricted will have permissive tools in production. A system where they're restricted unless granted will not.

Same principle for approval requirements, data retention, and logging verbosity: the safe state is the one you get by doing nothing.

Content policy and IAM

Two more items the blueprint lists here. Content policy — defining what the application will and won't produce, enforced at the output layer rather than hoped for at the prompt layer. Identity and access management — who can invoke what, with access levels verified at call time rather than assumed from the session.

Key concept

No single guardrail is sufficient — combine behavioural, detective, and preventive layers. Scope agent credentials to the task, because a manipulated agent can only use what it holds. And make the safe state the default state.

Practice scenario

ScenarioA team says their agent's broad production credentials are fine because it's internal-only.
Work it through, then open this

Internal-only describes network reachability, not what the agent does with the content it processes — and indirect injection arrives through documents and tool results, not the network edge. This is a least-privilege violation regardless. Scope the credentials to the task so a manipulated agent can only exercise what it genuinely needs.

Build exercise — Audit what your agent is actually allowed to do

Intermediate · 25 min

What you’ll learn

  • Finding least-privilege violations before an incident does
  • Checking that guardrails are layered rather than single
  1. List every credential and permission your agent holds, then list what its task genuinely requires. Compare the two.

    • Why: Agents launched by people tend to inherit their launcher’s access, and that is a finding before anything goes wrong.
    • You should see: At least one permission wider than the task — production write access is the usual one.
  2. Name which guardrail layer you have: prompt hardening, input filtering, output filtering, permission boundary. Count how many are real.

    • Why: Answers offering a single layer are wrong on that basis — each catches what the others miss.
    • You should see: Most systems have one or two, and it’s usually prompt hardening plus nothing enforceable.
  3. Check your defaults: if someone deployed this without changing a setting, are tools permissive or restricted?

    • Why: Most deployments never change defaults, so the default is the real posture.
    • You should see: The safe state should be what you get by doing nothing.

Exam traps

Offering one guardrail as a complete solution

Layers exist because each catches what the others miss. A single-layer answer is wrong on that basis.

Excusing broad credentials because the agent is internal

Internal-only describes network reachability, not what the agent does with content it processes.

Letting an agent inherit its launcher’s full credentials

A least-privilege violation before anything goes wrong. Scope to the task to bound the blast radius.

Relying on permissive defaults plus documentation

Most deployments never change defaults. The safe state has to be what you get by doing nothing.

Sources

Quick check

An agent authenticates successfully on every request while being manipulated into enumerating an entire patient table. Which control surfaces this?