The Secure Agent Playbook

A field guide for building and hardening systems where an AI agent does real work on a real computer — with access to files, tools, credentials, and the network — and where you cannot fully trust that the agent won't be tricked.

This repository is written for two readers:

• The engineer or team standing up such a system, or hardening one that already exists, who wants a method rather than a checklist.
• The owner, buyer, or decision-maker trying to understand what the problem actually is and whether a given approach is sound. Start with this page; it is written for you, in plain language.

What this is about, in plain language

An AI agent is useful precisely because it is flexible: you describe what you want in your own words and it figures out how to do it. That same flexibility is what an attacker goes after. The attack is rarely a clever exploit against the software — it is a few sentences of instruction hidden inside something the agent reads: a document you upload, a web page it fetches, an email it summarizes. The agent is persuaded to act against your interests, and from the software's point of view nothing illegal happened at all.

So the problem this playbook addresses is not "how do we make the agent impossible to trick." That problem has no solution. The problem is:

Assume the agent can be tricked. Build the system so that a tricked agent still cannot reach your secrets, cannot take a consequential action without a human, and cannot erase the record of what it did.

Everything here follows from that one assumption. It is the difference between a system whose safety depends on the agent always behaving, and a system whose safety holds even when the agent misbehaves — because the dangerous capabilities were never in the agent's reach to begin with.

The specific problems it addresses

• A capable agent with real access. Not a toy in a web sandbox — an agent that can write files, run programs, and reach services on a machine someone depends on.
• Untrusted input arriving through normal channels. The uploaded file, the fetched page, the pasted text. The agent has to read these to be useful, and any of them can carry hidden instructions.
• Secrets the agent needs work done with, but must never hold. API keys, passwords, tokens. The agent has to use them without ever being able to read them — because anything it can read, an attacker who has steered it can exfiltrate.
• Actions that leave the system. Sending data out, spending money, publishing. These need a person's explicit say-so, and the agent must not be able to grant that say-so to itself.
• An honest record that survives a bad day. If something goes wrong, you need to know what happened — and the record has to be somewhere the compromised machine could not quietly rewrite.
• Proving any of it is true. A configuration file that claims the right thing is not a system that does the right thing. The hardest part of this work is verification, and it has its own section here.

The methodology, in six moves

These are expanded in northstars/ and made concrete in playbooks/. In brief:

1. Assume compromise, then contain. Design for the agent already being under an attacker's influence. Safety comes from containment, not from good behaviour.
2. Split the trust boundary out before hardening anything. You cannot secure a single program that both holds the secrets and reads the untrusted input. The first move is structural: separate those jobs into different processes and accounts with one narrow door between them.
3. Least authority, everywhere. Each component can do only its one job. No single component, if compromised, unlocks the others.
4. A human gates anything that leaves. The agent can request a consequential action; only a person can authorize it, and the authorization happens somewhere the agent cannot reach.
5. A tamper-evident record, kept off the machine. Write it before the action, append only, and ship it somewhere the machine that produced it cannot alter.
6. Prove it, don't promise it. Every protection gets a test that is watched failing before it is trusted to pass — on the real system, by observed behaviour, not by reading a config file.

How the repository is organized

• northstars/ — the guiding principles. The why behind every decision. Read these first; they are short, and they make the rest make sense.
• playbooks/fresh-build/ — building a secure system from scratch: the order of operations and the concrete mechanisms.
• playbooks/hardening-existing/ — taking a system that already exists and making it safe, which is a different and in some ways harder job.
• security-reviews/ — how to review the result after the hardening, including the discipline of not breaking the thing you are inspecting, and what "done" actually means.
• reference/ — a glossary, a threat catalogue, the recurring failure patterns worth recognizing on sight, and pointers to public standards for further study.

In one sentence

Assume the agent can be compromised, and build so that even then your secrets stay sealed, nothing consequential leaves without a person's say-so, and the record of what happened cannot be quietly altered — and prove each of those claims with a test you have watched fail.

License

Licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). Use, adapt, and redistribute it — including commercially — with credit to fisho0 and cl_1. See LICENSE for the full terms.

This playbook is distilled from the hands-on work of designing, building, and adversarially reviewing one such system end to end. It is written as a general method; the specific products, hosts, and names from that build have deliberately been left out so the method stands on its own.

North Stars

These are the guiding principles — the reasoning that every concrete decision elsewhere in this playbook traces back to. They are deliberately few, and deliberately stated as properties to hold rather than steps to follow, because a list of steps always has a gap shaped like the step nobody wrote down.

Read them in order. Each is short.

1. Assume compromise, and contain it — the core stance that makes everything else coherent.
2. Split the trust boundary out first — why the first move is structural, not a patch.
3. Least authority, and no shared fate — one job per component; one compromise never unlocks the rest.
4. Capability over instruction — the agent is free inside its box and crosses a narrow, typed boundary to do anything outside it.
5. A human gates what leaves; the record survives the box — the two controls that hold even after a successful attack.
6. Prove it, don't promise it — a protection you have not watched fail is decoration.

A seventh principle governs who you are building for rather than how you build, and it silently decides whether the other six are any use:

7. Build for the person who will never read this — if a control requires its user to understand security, the control is what changes, not the user.

The shortest version

Assume the thing you are protecting is already compromised. Define the property you need to hold in spite of that. Write a check that fails until the property holds. Prove it on the real system — never from the config, never from the happy path, never only once.

1. Assume compromise, and contain it

The stance

An AI agent is powerful because it is flexible, and flexibility is exactly the surface an attacker works against. The attack does not usually look like an exploit. It looks like instructions hidden in the data the agent was asked to read — a document, a web page, a pasted message. The agent follows them because following instructions in its input is what it does.

You cannot reliably train or prompt this away. Rephrasing, encoding, splitting an instruction across turns, or burying it in a file all defeat a filter that was looking for the previous phrasing. So the design does not try to make the agent impossible to trick.

The design assumes the agent can be tricked, and makes a tricked agent harmless.

A tricked agent is harmless when the dangerous capabilities were never within its reach: it cannot read the secrets, it cannot take a consequential action without a human, and it cannot erase the record of what it attempted. Every other principle in this playbook exists to make that assumption safe to hold.

Why detection is not the primary control

It is tempting to put a classifier in front of the agent that inspects each prompt for malicious intent. This has a place — for alerting, for after-the-fact review — but it must never be the thing standing between an injected instruction and a destructive action, for two reasons:

• It is bypassable by construction. The space of ways to phrase a malicious request is unbounded; the classifier only knows the ones it was trained on.
• Its failures are invisible. A classifier that quietly misses an attack gives you no signal that it missed. By this playbook's own rule — a check you have never watched fail is decoration — an undetectable miss is worse than no check, because it manufactures confidence.

Detection is a layer. Prevention is a structure. When you find yourself reaching for a smarter detector, ask instead what capability you could remove so that the detection would not have mattered.

The practical consequence

Every design review asks one question first: if the agent were fully under an attacker's control right now, what could it do? The answer to that question — not the answer to "how likely is the agent to be compromised" — is the measure of the system's safety. A system whose safety depends on the agent behaving well has no safety at all; it has a hope.

2. Split the trust boundary out first

You cannot harden a thing that is also the thing holding the secrets

The most common mistake is to take a single program that does everything — reads untrusted input, holds the credentials, talks to the outside world, writes the audit log — and try to make that program secure. It cannot be done, because the attack surface and the crown jewels live in the same process. One successful injection and the attacker is already past every internal check, standing next to everything worth protecting.

So the first move in both a fresh build and a hardening job is not a patch. It is structural separation:

• The part that reads untrusted input and decides what to do (the agent) is split from
• the part that holds the secrets and actually performs privileged actions (call it the broker), and
• a single narrow door connects them, through which only a small, fixed vocabulary of typed requests can pass.

Once that split exists, every security property becomes a property of one small, single-responsibility component rather than a property of "the whole application." You are never again hardening a monolith; you are proving one fact about one named surface at a time. That is what makes all the downstream work tractable.

The question to ask before anything else

What is the smallest thing that must be trusted — and can I make it a separate process, account, and directory tree, with exactly one entrance?

If the secret and the attack surface still share a process after you answer that, nothing you build downstream will hold. Do the separation first.

What the broker is, and is not

The broker holds the credentials, performs the privileged operations, and writes the record. Crucially:

• It accepts structured, validated requests, never a command string. There is no sentence the agent can compose that the broker will execute as an instruction — only fields it will validate against a schema and refuse if unknown.
• It has no "give me the secret" operation at all. Not a disabled one — a nonexistent one. A capability that does not exist cannot be abused, logged by accident, or argued into existence by a persuasive prompt.
• It is the only component that reaches the outside world, and only to a short, fixed list of destinations the agent cannot change.

The broker is small on purpose. Everything you add to it, you will have to trust. The discipline is to keep it boring.

3. Least authority, and no shared fate

One job per component

Split the system into separate accounts and processes, each able to do only its one job, so that compromising any one of them does not hand an attacker the others:

• The account that builds the software cannot run it in production.
• The account that runs the user-facing service cannot reach the network or read the vault.
• The component that holds credentials is isolated from the component that holds business data, which is isolated again from the component that issues the system's own internal tokens.

The result is that there is no single account which, if breached, unlocks everything. An attacker who lands in one component finds themselves in a room with one door, and that door leads only where that component legitimately needs to go.

No shared fate

The phrase to keep in mind is shared fate: two things share fate when compromising one compromises the other. The goal is to remove shared fate wherever a single breach would otherwise cascade.

The sharpest example is the audit record. A log that lives on the machine it is logging shares that machine's fate: an attacker who owns the machine owns its history and can edit it to hide the intrusion. The record must therefore leave the box (see principle 5). The same reasoning applies to credentials, to backups, and to the system's root of trust.

The least-authority questions

For every component, ask:

• What is the one thing this needs to do? Grant exactly that, and nothing adjacent "for convenience."
• What does it need to reach? If the answer is "the network," ask which host and which port — an allowlist of one address beats "outbound allowed."
• If this were fully compromised, what is the blast radius? If the answer includes another component's secrets or data, the separation is not finished.

A trusted network is not a security model

A recurring failure is to treat "it's only reachable on our internal network" as a boundary. It is not. The client is never trusted, internal or not. A gate whose secret or whose protected content lives in the page the client receives is theater. The one test that settles it: an unauthenticated request returns nothing — no data, no partial render, no "empty but 200." If a request without credentials can obtain anything, the control is decorative regardless of what network it sits on.

4. Capability over instruction

Freedom inside the box; a narrow door out

A naive reading of "limit what the agent can do" leads to a system where every action is a permissioned verb the agent must request — which strangles the agent's actual work and loads a whole permission vocabulary into its reasoning. That is the wrong shape, and it is worth stating why so nobody rebuilds it.

The resolution is that a permission is not a way of thinking; it is a boundary crossing.

• Inside its sandbox, the agent is free. It can run programs, compile, edit, iterate — no verbs, no asking, no permission vocabulary. Freedom inside is the entire point of the box, not a concession. The sandbox is built so that this freedom cannot matter: no network, no access to the host, nothing of value within reach (see sandboxing untrusted code).
• A verb exists only where an action affects something outside the box. Make something reachable; send something to a customer; spend money; read the central knowledge store; publish. These are the boundary crossings, and there are only ever a handful of them.

If the list of verbs ever needs to grow to fifty, the boundary has been drawn in the wrong place. The small verb count is itself a design check.

A useful precedent is a coding agent that runs shell commands freely in its working session and asks only when it wants to do something beyond its granted permissions. A verb is that ask, formalized. The agent carries no vocabulary; it does ordinary work and occasionally knocks on the one door.

Capability is chosen before the content is read

This is the principle that defines prompt injection out of existence at the boundary:

The agent's capability set is decided by the channel and the identity of the request — never by its content.

If what arrives in the request can influence what the agent is permitted to do, you have already lost, because content choosing capability is the definition of injection. A request that comes in through a low-trust channel gets a low-trust tool list, and no sentence inside that request can upgrade it. The decision is made first, by code, before a single byte of the content is interpreted.

The gate is deterministic code the model cannot reach

When a boundary-crossing verb is requested, the thing that decides whether it is allowed is plain, deterministic code — and it is not in the model's context, not in a prompt, and not in any file the agent can write.

• A policy the agent can read is a policy it can be argued out of.
• A policy the agent can edit is not a policy.

If a rule can be weakened by persuasion, it was never a gate; it was a request. This is the control that silently inverts the moment someone makes the agent "more capable" by letting it construct its own calls — every such convenience puts the decision back inside the model. Name it and guard it, because the erosion looks exactly like a feature.

5. A human gates what leaves; the record survives the box

These are the two controls that still hold after a successful attack — after the agent has been fully steered by an adversary. Everything else reduces the odds of reaching this point; these two bound the damage once it is reached.

A human gates anything that leaves or is irreversible

Reads are free. But any action that sends data out of the system, spends money, or cannot be undone happens exactly one way:

• The agent can request the action. It cannot perform or approve it.
• The request surfaces to a person, and that person's own confirmation — their click, in their own session, against a token the agent never sees — is what authorizes it.
• The action is then carried out by the isolated component that actually holds the capability, and the whole sequence is written to the record.

The human is the gate for anything that leaves the box, and the agent cannot open that gate by itself no matter what it was told to do. This is also the single most load-bearing control for accountability: it is the demonstration that a person remained in control of every consequential act — a materially different practical posture from "the software did it." (That is an engineering observation, not legal advice.)

One design note learned the hard way: a gate placed after a teardown converts a refusal into an outage. If the step that can refuse an action runs after you have already torn down the running service, then a legitimate refusal takes the service down with it. The property to hold is that anything that can refuse has been exercised, on the real artifact, outside the window where a refusal would cause harm. Order your checks so the ones that can say no run before anything irreversible happens.

The record is written first, appended only, and shipped off the box

Every meaningful action is written to an append-only record — entries can be added, never silently changed or deleted:

• It is written before the action runs, so no action can happen without a trace.
• Even an administrator's change is itself a recorded event, never a silent rewrite.
• The path that writes the record and the path that reads and manages it live in separate trust domains, so the component that produces evidence cannot tamper with the evidence it already produced.

And the record leaves the machine. A black-box recorder must not share the fate of the thing it records: if the main system is ever compromised, its own history must not be editable to hide the compromise. Concretely, the box can write to the off-box store and cannot delete from it — that one-way property is the whole point.

A detail that matters: the off-box store need not hold anyone's business data, only the tool-call record. This keeps it cheap to run centrally and keeps the sensitive data on the owner's own machine, which is itself a security decision (see principle 3 and least authority).

6. Prove it, don't promise it

This is the spine of the whole method. Every protection gets an executable proof, and you watch it fail before you trust it to pass.

A check that has never gone red is not a check

A green result from a check you have never seen go red tells you nothing. It might be testing the right property; it might be structurally incapable of ever failing. You cannot tell the two apart from a passing run.

So, before trusting any new check:

Name the input that should make it fail. Feed it that input. Watch it go red. Then fix the system and watch it go green.

This sounds like extra work and is in fact the cheapest insurance in the whole process. The alternative — a check that was green from birth — is how a protection that never actually worked sits in place for months, radiating false confidence, until the day it was supposed to matter.

Prove the property on the real system, by behaviour

A configuration file that says the right thing is not a system that does the right thing. Again and again, a fix is accepted by reading the config and turns out to be inert: an environment variable ignored by the installed version of a tool, a setting that applies only to new objects and silently skips the existing one, a resource ceiling defeated by a mechanism nobody accounted for.

Two questions close most of this:

• The counterfactual: Would I have observed this exact result if the change had never been made? If yes, you have a result, not evidence. Read the reason a check passed or failed, never just its exit code — a status code is a bucket that several different causes fall into, and only one of them is usually your claim.
• The behaviour, at the real version, on the real host: exercise the actual thing and read the kernel or the service back — not the file you wrote.

Prove the bytes that actually run

A proof that reads the developer's working copy proves the wrong bytes. The thing that executes in production may differ from the thing in the editor — a stale checkout, an un-synced reference, a file bind-mounted over the one you are reading. The question is never "is the source correct"; it is "are the bytes that actually execute the ones I verified." Verify against what the system actually serves, identified by content hash, not against your local copy.

Honest states, including "I cannot tell from here"

A check must be able to report a fourth state beyond pass, fail, and absent:

I cannot judge this from here.

A check running on a machine cannot see a control that protects the machine from outside — a provider firewall, for instance, is not visible to the host it protects. From inside, "open to the world" and "allowlisted to one peer" look identical. A check that claims either one from that vantage is lying with confidence. The honest behaviour is to state what it can see, name the control it cannot, say who could verify it, and refuse the conclusion. That state must not trip an alarm — an honest "unverifiable" treated as a failure trains people to ignore the failures that matter.

Know when to stop

A proof exists to retire a doubt. When the doubt is retired, the proof is done. Building a larger proof of an already-proven thing is motion, not progress. Before each new proof, ask what doubt it retires that the last one did not — and if there isn't one, stop and ship.

7. Build for the person who will never read this

This principle is about who you are building for rather than how you build, and it silently decides whether the other six are any use. A perfectly abstracted, fully tested, provably sound mechanism that the actual user cannot operate is a failure that passes every other check in this playbook.

The north star, and it is falsifiable

If a mechanism requires a non-technical person to learn a security concept in order to use the product, the mechanism is what changes — not the person.

Hold any design up against that sentence. The user should be able to say what they want, in their own words, and get it — seeing at most one plain instruction, once, and never having to understand the machinery underneath.

The four slots every mechanism must fill

A security mechanism is unfinished until you can answer all four:

1. What does the user do? Ideally: say what they want, in their own words.
2. What do they see? Ideally: nothing. At most one plain instruction, once.
3. What do they never have to know? Name it explicitly — so that six months later nobody leaks it into a UI label, an error message, or a settings page.
4. What does this mechanism refuse, and what proves it refuses? The control itself, named in the same breath as the experience.

The fourth slot is the one that makes this safe. Without it, the first three can be satisfied by deleting the control — the most invisible security is no security, and an agent optimizing for "the user sees nothing" gets there fastest by removing the gate. The fourth slot closes that door, because a question about what a mechanism refuses cannot be answered by a mechanism that refuses nothing.

The ordering that must never invert

• The threat model decides whether a control exists.
• This principle decides how the user meets it.
• If the only way to make the experience seamless is to weaken or drop a control, that is a design problem to escalate — never a license. The answer is almost always a better interface, not a smaller gate.

A caution for anyone writing this down

Do not lead a section with a worked example. An example at the head of a requirement becomes the specification: people implement the one story instead of the general property. State the property first; put the example underneath, labeled as one instance of it, so that the next person fills the requirement for their own feature instead of copying someone else's.

Playbooks

Two jobs, related but distinct:

• Building a secure system from scratch — you control the order of operations, so you do the structural moves first and never have to retrofit them.
• Hardening a system that already exists — the structure is already set, often wrong, and you have to find what actually runs before you can change it safely. In some ways the harder job.

Both rest on the north stars; read those first. The reviews that come after either job are in security-reviews/.

The shortest version of the whole method

1. Split the trust boundary out before hardening anything.
2. Every property gets a proof you watch go red first.
3. Prove the bytes that actually run, on the real machine, by behaviour — not by config.
4. Find the class of defect, not the single bug — ask the property, and trace the real operation to find the siblings a text search cannot.
5. Reproduce the build from zero on a second machine; demand the two agree.
6. Let a blind adversary attack your claims.
7. The operator never holds the secret; a rotation is proven by the old secret being refused.
8. Report honest states; "unverifiable" never rounds up to "fine."
9. Write down each decision as you make it; re-ground from the record after any interruption.
10. Stop when the doubt is retired.

Building a secure system from scratch

The advantage of a fresh build is that you control the order. Do the structural moves first and you never have to retrofit them against a running system.

The order below is not arbitrary. Each step depends on the one before it: you cannot grant least authority before the components are separated; you cannot sandbox untrusted code usefully until there is a boundary for it to sit behind; you cannot prove the record is tamper-evident until there is a record.

1. Model the threat and draw the trust boundary — decide, in writing, what you are defending against and where the line is.
2. Split before you harden — the structural separation that everything else stands on.
3. Separate accounts, least authority — one job each, no shared fate.
4. The credential broker — how the system uses secrets it can never read.
5. Egress control — an allowlist at the door out, enforced by the kernel, not the application.
6. Sandboxing the code the agent runs — a disposable, networkless, unprivileged box where the agent is free precisely because it cannot matter.
7. The human gate and typed verbs — the one door out for anything consequential.
8. The append-only, off-box record — evidence that survives a compromise.
9. The supply chain — controlling what third-party code gets in.

Encryption and real authentication belong from the first commit, not as a later hardening pass. Retrofitting transport security and sign-in onto a system that grew up without them is far more error-prone than building on them from the start.

1. Model the threat and draw the trust boundary

Before a line of code, write down two things: what you are defending against, and where the trust boundary sits. Skipping this is how a team ends up with a list of controls and no way to ask whether anything is missing — a control list cannot tell you what it misses; a threat list can.

Name the threats, then map controls to them — in that order

It is natural to start from the controls you know how to build. Resist it. Start from the threats, map each to a control, and the gaps become visible. For an agent with real access, the recurring threat categories are:

• Prompt injection. The attacker is content — a file, a page, a message the agent reads and is steered by. Mitigated by limiting what the agent can do regardless of what it is told (capability limits, sandbox, the human gate).
• A malicious user or guest. The person is the attacker — editing files directly, or asking the agent to build something harmful. Mitigated by limiting blast radius: if it is the user's own machine and their own data, wrecking it is their problem; the parts that matter are whether they can reach other users or you, and whether they can use the machine to attack third parties. Those are egress and isolation problems, solved with firewalls and tenancy.
• Privilege compromise. The agent or a component gaining authority it should not have. Mitigated by account separation and the broker.
• Tool misuse / a tool that impersonates a real one. Mitigated by a signed, versioned registry of the tools and skills the agent may use.
• Goal manipulation and deceptive behaviour. The honest limits of the technical controls. You can bound an agent's actions; detecting that its goal has been changed is largely beyond a gate, and the real answer is attribution and audit plus a human on anything outward. Say this plainly rather than claiming a defence you do not have.

A category worth watching for is the one your own product's shape creates, because a generic threat list will never flag it. A feature that ingests the user's own content and distills it into standing knowledge the agent later trusts, for example, carries risks of its own — and hardening a persistent memory or knowledge store is a distinct problem this playbook does not cover. Only modelling your system, rather than a borrowed list, surfaces the threats that are specific to it.

The two threats need opposite controls

A frequent confusion is treating "the agent is tricked" and "the user is malicious" as one problem with one fix. They are opposite:

• Against a tricked agent, you constrain what the agent can do.
• Against a malicious user, you constrain blast radius.

A single control cannot cover both, and the hardest real case is the overlap: a user plants content that the agent later ingests as instructions. The resolution is that it does not matter what the content says if the agent cannot act outside its sandbox — but it means content from any user-writable surface must be treated as untrusted input, on the same footing as a fetched web page, never as configuration and never as instruction.

Write down the scope decision

"Who are the first users?" is a containment decision, not a detail. A system sold to known, contractually-bound businesses is a genuinely different threat model from one open to the anonymous public, and it is legitimate to make attribution and audit a primary control in the first case. State the scope explicitly so nobody designs for the public case before it is real — and so nobody quietly relaxes a control on the assumption that the public case never arrives.

2. Split before you harden

This is north star 2 applied as the first build step. Expanded here into the concrete shape.

The shape

Three responsibilities that a naive build fuses into one program, separated into three components:

• The agent / front end. Reads untrusted input, reasons, decides what it wants to do. Holds no secrets. Reaches nothing privileged directly.
• The broker. Holds the credentials, performs privileged operations, writes the record. Owned by the most privileged account; accepts only structured requests through one entrance.
• The sandbox. Where code the agent writes actually runs — disposable, networkless, unprivileged (its own section follows).

The agent talks to the broker through a single door that carries a small, fixed vocabulary of typed requests. It never talks to the outside world, the credentials, or the record directly.

Why "one connection at a time, structured input only" matters

The broker takes structured input — fields it validates against a schema — never a command string. This is not a style preference; it is the property that makes injection inert at the boundary. There is no sentence an attacker can get the agent to emit that becomes an instruction to the broker, because the broker does not interpret sentences. It validates fields and refuses unknown ones.

Build the skeleton first, with the boundary real from day one

Stand up the real structure — real transport security, real sign-in, the real agent/broker/sandbox split — with placeholder functionality behind it, before building any feature. A feature built on a boundary that is "coming later" tends to grow tendrils across the boundary that are painful to sever afterwards. Encryption and authentication from the first commit; the split from the first commit; features after.

The test you can apply immediately

Once the split exists, you can ask of any new feature: does this require the agent to hold a secret, reach the network directly, or write the record itself? If yes, the feature is reaching across the boundary and the design is wrong — route it through the broker as a typed verb instead. If that would need a fiftieth verb, revisit where the boundary is drawn (see capability over instruction).

3. Separate accounts, least authority

The split from the previous step becomes real only when the components run as different operating-system accounts, each granted exactly its one job. Separation that exists in the code but not in the account model is not separation; a single process compromise crosses it.

The accounts, as distinct trust domains

A typical split, with the principle rather than the exact names:

• A build account — assembles the deployable artifact. Can write the build tree; cannot run the production service, cannot read the vault, cannot reach the record.
• A serving account — runs the user-facing service. Cannot reach the network beyond what it serves, cannot read the vault.
• A broker account — holds credentials and performs privileged operations. The only account that reaches external destinations, and only allowlisted ones.
• A sandbox-runner account — spawns the disposable sandboxes. Denied the network at the account level, so even a bug in the runner cannot reach out.
• A record account — manages the audit store. Separate from whatever writes the record, so the producer of evidence cannot alter it.

Each is a wall. An attacker who lands as the serving account cannot become the broker account by reading a file, because the file that matters is not readable by the serving account.

The code that runs as root must be held by root

A subtle and high-value rule: when a privileged process (an installer, a deploy step) executes, sources, or reads-into-an-interpreter a file, that file and every parent directory of it must be owned by the privileged account and not writable by any lesser account. Otherwise a lower-privileged account can swap the file — or replace a parent directory with a symlink — and have its own content executed as root.

The general rule, which is mechanically checkable and therefore should be a check:

A privileged mechanism must not point at a path a less-privileged user can write or replace. For every such reference, assert that the leaf and every parent is owned by the privileged account and not group- or world-writable, or the installer refuses.

This covers more surfaces than it first appears: a privileged scheduled-task target, a system service's executable path, a configuration file read at startup, a directory a privileged process writes into. The resolution shape to aim for is the parents belong to the privileged account, the leaf may be shared — two accounts can share a working file through a group without either being able to swap the path out from under the other.

Do not let a privileged process act on a resolved path it did not create

A specific, expensive instance of the rule above. A root installer that resolves a symbolic link and then changes ownership of the result can be tricked: plant the link pointing at a sensitive system directory, and the installer hands that directory to a lesser account. The hardening against "does this walk follow links" does not help, because the escalation enters through where the walk starts. The rule: a privileged installer must never change ownership of a path it did not itself create — create, then set ownership, never set ownership of whatever happens to be there.

4. The credential broker

The system does real work with secrets — API keys, passwords, tokens — that the agent must never be able to read. The mechanism that makes this possible is a broker: a single-purpose component that folds a credential into an operation inside its own process and returns only the operation's result.

The one hard rule

The agent (and any component that an attacker might reach through the agent) never holds the credential value. The broker uses it on the agent's behalf and hands back only output.

Concretely, the broker exposes operations like "run this query," "make this request to this allowlisted host," "sign in over this connection" — and performs them, with the secret supplied from its own storage. There is deliberately no operation that returns a secret's value. Not a disabled one — a nonexistent one. An operation that prints a secret to its output would print it into whatever reads that output: a log, a transcript, a model's context. The capability must not exist.

Inspection operations exist, but they return metadata only: that a secret exists, its length, which keys are present, what an item references. Never the value.

No secret value in a command line, ever

A command line is observable: by the process table, by process accounting, by audit logs, and by anything reading the output of a tool that ran the command. A secret passed as a command-line argument is a secret leaked.

The discipline that actually works is to ask the property, not to check against a list of known-bad flags:

Does the secret value end up in this process's argument vector?

A list of bad forms always has a hole shaped like the form nobody listed. The classic escape is a credential embedded in a URL (scheme://user:password@host) — it does not look like a password flag, so a check that enumerates flags sails right past it, even when run by someone actively hunting for violations. Ask the property every time.

Safe ways to hand a secret to an operation, none of which touch the argument vector:

• An environment variable — visible to the process and root, not to the process table.
• Standard input — pipe the value in.
• A mode-restricted temporary file — write it with tight permissions, pass the path, shred it after.
• A native credentials file the tool reads on its own (the conventional per-tool config).

If a tool offers no way to take a secret except on its command line, the fix is to add one that reads from the environment — not to improvise the value into the command line.

Rotation is proven by the old secret being refused

When a credential is rotated — routinely, or because it may have leaked — the proof that rotation worked is that the old value is now refused, not that the new value works and not that the service came back up. And read the reason for the refusal: an expiry error is not a revocation, because the credential would have expired anyway. Where a credential has a short lifetime, test the revocation first, before the clock can do your work for you and hand you a misleading pass.

And record the exposure where the credential itself lives, not only in a chat log or a ticket: a disclosure noted only in a transcript dies with the transcript, and the next person to handle that credential must be able to see that it happened. Recording it on the item is what turns an incident into a control.

5. Egress control

The single most effective limit on a compromised agent is that it cannot reach out. An agent that cannot open an outbound connection cannot exfiltrate data, cannot phone home, and cannot use the machine to attack a third party — no matter how thoroughly it has been steered.

An allowlist, not a denylist

Default-deny outbound, and permit exactly the destinations the component legitimately needs — by address and port, not by "the network." A denylist of known-bad destinations is a losing game; an allowlist of the two or three places a component actually talks to is both stronger and easier to reason about.

A concrete pattern that ports well: per-account outbound firewall rules keyed on the OS account, permitting (for example) only the local model endpoint and the one external service the component needs, and dropping everything else. The agent's build sandbox gets an allowlist that reaches only the model endpoint and the single data store it needs; the serving account reaches nothing outbound at all.

Enforce it in the kernel, below the application

The allowlist must live where the application cannot change it — in the host firewall and in the account's own network restrictions — not in application configuration the agent could influence. Two independent fences are better than one: a firewall rule and a per-unit address restriction, so a mistake in either still leaves the other standing.

Know what a blocked connection looks like

When you verify egress control, know the difference between the failure modes, because they mean different things and a test that confuses them proves nothing:

• A kernel drop produces a timeout — the connection attempt hangs and eventually gives up with no response.
• A rejection produces an immediate, explicit refusal.
• A genuinely unreachable host produces yet another distinct error.

A test that asserts "the connection failed" without distinguishing these can pass for the wrong reason — the host was simply down, say, rather than the control working. Ask the operating system why the connection failed (the specific error), and assert the specific cause you intended, not merely that something went wrong.

Verify from both sides

A host cannot fully see the control that protects it from outside. A provider-level firewall is not visible on the machine it protects: from inside, "reachable by the world" and "reachable by one allowlisted peer" look identical. So verify egress and ingress controls from the other side as well — attempt the connection from a machine that should be blocked, and confirm it is — and where you genuinely cannot see the control from your vantage, report that honestly rather than asserting a pass you cannot support.

6. Sandboxing the code the agent runs

When the agent generates and runs code, that code executes inside a box built so that whatever the code does, it does in a sealed room that is torn down when it finishes. This is what lets the agent be free inside — running programs, compiling, iterating — because the freedom cannot reach anything that matters.

The properties the box must have

• No network. The sandbox cannot open an outbound connection. Confirmed at the account and namespace level, not merely configured.
• No privileges. It runs as an unprivileged user inside the container, with all extra capabilities dropped and a system-call filter in place.
• No package manager, and no registry it could reach. Nothing to pull new code in at run time. The image is built ahead of time, pinned by content digest, and contains only what the workload needs.
• A read-only root with one small writable scratch area; the work mounted read-only.
• Resource ceilings — memory, process count, CPU — read back from the kernel before the job starts and refused if they did not actually take effect.
• No persistence. Nothing the code creates outlives the job. The box is destroyed and recreated per run.

Disposability is the deciding property

When you choose an isolation mechanism, compare them by what an escape costs the attacker — that is the only thing isolation buys. But the property that matters most is not the strength of the wall; it is disposability.

A sandbox destroyed and recreated per job means a compromised sandbox has a lifetime measured in minutes. That is worth more than an extra kernel boundary, because the realistic attack is "the agent got talked into running something," not "the agent found a novel kernel exploit." A disposable container that the attacker cannot persist in beats a stronger-walled environment that lives for weeks.

A practical ordering, learned from doing it:

• Start with disposable unprivileged containers, running rootless, as a non-root user inside the container (that gives you a jailed user and a container together). They start in a second, you can debug them trivially, and the full tooling ecosystem is available while you are still discovering what the sandbox actually needs to do.
• Treat a stronger VM-level boundary (a microVM) as a later hardening step, ported to once the workload is settled. Its design center is "boot fast, run one thing, die" — which fits disposability well — but you pay for it in image maintenance, manual networking, and debuggability, and those costs are worst exactly when the workload is still undefined. Port a known workload to it; do not build its images for a workload you have not pinned down.

The deciding argument is not which boundary is theoretically strongest. It is that you cannot harden a sandbox well until you know what it has to do, and you learn that fastest in the cheap, debuggable, disposable one.

Prove the sandbox from inside its own namespace

A sandbox's isolation claims must be proven from inside the sandbox's namespace, not from the host's view. A mount namespace can make the host's view of a file lie about its content exactly as it lies about isolation — the file you read from the host may not be the file the sandbox sees. Run the proof in the same namespace the real job runs in, exercise the real restriction (try to open a network connection and watch it fail; try to exceed the memory ceiling and watch the kernel stop it), and read the result back from the kernel.

7. The human gate and typed verbs

This is north star 5 made concrete: the one door through which anything consequential leaves the system, with a person's explicit confirmation required to open it.

The mechanism

• The agent emits a request to use a verb — a structured object, not a command. The verb names a boundary-crossing action: make this reachable, send this to a customer, spend this, read the central store, publish this.
• The request returns pending. The agent's tool, by construction, cannot perform the action and cannot approve it. It can only ask.
• The request surfaces to the owner, who confirms it by a click in their own session, against a token the model never sees. The token is the proof that a human — not the agent — authorized the act.
• The action is then performed by the component that actually holds the capability, and the whole sequence is written to the record.

The verb registry — the list of verbs and their policies — is held by the most privileged account, where the agent cannot read or edit it. A policy the agent can read is a policy it can be argued out of; a policy it can edit is not a policy.

Verbs are carried by skills, so the agent carries no vocabulary

A real objection to "permission for every action" is that it loads a permission vocabulary into the agent's reasoning and stops it working naturally. The resolution: a skill is the carrier for a verb. The agent does normal work inside its sandbox; when a situation calls for a boundary crossing, it reaches for the relevant skill, and the skill is what carries the verb. The agent memorizes nothing.

This has a security consequence worth stating: editing a skill edits a permission boundary. That is exactly why the registry of skills and tools must be signed and versioned (see the supply chain and tool misuse in the threat model) — so that changing what the agent is permitted to do cannot be a silent act.

What the user sees

A boundary crossing should present as one brief, plainly-administrative notice — something was done here on purpose — naming no mechanism, followed by the agent explaining in its own words what happened and what is next. The agent is prompted with what happened, never with any held secret. Every harness intervention — a held message, a refused verb, a confirmation request — follows this same shape. A gate that announces itself as a system error has leaked its own plumbing into the one place principle 7 forbids.

Order the gate before the irreversible step

A gate that can refuse an action must run before anything irreversible happens. If you tear down the running service and then run the check that can refuse, a legitimate refusal has become an outage. State the property as: anything that can refuse has been exercised, on the real artifact, outside the window where a refusal would cause harm. Where a check genuinely must inspect the finished artifact and so cannot run before the teardown, make the teardown cheap to reverse (a pre-staged rollback), so a late refusal costs seconds, not an outage.

8. The append-only, off-box record

Every meaningful action is written to a record before it happens, in a form that can be appended to but not silently changed, and shipped to a machine the producer cannot alter. This is both a security control and the thing that makes a fleet operable — the same record that would show an intrusion is the one that lets you run the system at all.

The properties

• Written before the action. The record entry is created before the action executes, so no action can occur without a trace. If the system dies mid-action, the intent is still recorded.
• Append-only. Entries can be added, never edited or deleted in place. Even an administrator's change is a new recorded event, not a rewrite of an old one. On a typical filesystem this is enforced with an append-only attribute on the store plus restrictive permissions, so that even the writing account cannot rewrite history.
• Writer and reader in separate trust domains. The path that writes the record is distinct from the path that reads and manages it, so the component producing evidence cannot tamper with the evidence it already produced.
• Shipped off the box. The record leaves the machine that produced it. The machine can write to the off-box store and cannot delete from it. If the machine is compromised, its history is beyond the attacker's reach.

The one-way property is the whole point

A black-box recorder must not share the fate of the thing it records. State the design goal as: the box can add to its own history and can never edit it. The off-box store accepts appends from the box and refuses deletions and rewrites. Verify this the way you verify every control — attempt a deletion as the box's own identity and watch it be refused.

A concrete shipping shape

One workable design: each event is a small, self-describing record with a sequence number; the shipper asks the off-box store for the highest sequence it already holds and sends only what is newer; the store validates that sequence numbers arrive contiguously, so a gap or a replay is detectable. The connection to the store is mutually authenticated and pinned, and the store admits only the specific senders it has been told to expect. A tamper check on the sending side notices if the local record has fallen behind what the store already holds — which would indicate the local copy was rolled back.

Keep the off-box store minimal: it holds the tool-call record, not anyone's business data. That keeps it cheap to operate centrally for a whole fleet, and keeps the sensitive data on the owner's own machine where it belongs — which is itself a security and liability decision.

Do not confuse "the record exists" with "the record is faithful"

A record is only as good as the guarantee that it reflects what actually happened. Two failure modes to guard against: an action that happens without a record (the write was skipped or failed silently), and a record that claims an action that did not occur. The first is prevented by writing before acting and failing the action if the write fails — never letting a failed record write pass silently. The second is prevented by having the record written by the component that performs the action, not by the agent that requested it.

9. The supply chain

Third-party code is a common attack route, and an agent that can install packages widens it. The supply chain needs the same assume-compromise posture as everything else: treat an incoming dependency as untrusted until it has earned otherwise.

Pin everything; build from locked records

• Dependencies are pinned to exact, verified versions and rebuilt from locked records — never "whatever is newest today." A build that resolves "latest" at build time is a build whose contents you do not control and cannot reproduce.
• A transitive dependency you rely on but did not declare is a dependency you did not choose. If you depend on it, declare it; if it vanishes from under its parent, your build should fail loudly rather than silently losing a behaviour.

Screen new packages before they are allowed in

• Age-gate new packages: a package published minutes ago is in the window where live malware is most often caught and pulled. A short waiting period costs little and closes that window.
• Screen against known-malicious databases before a package is permitted.
• Disable install-time scripts. A package's install hooks are code that runs on your build machine with your build account's authority; default them off.

Wrap this in a single install path that every agent and every build uses, rather than leaving each one to call the raw package manager. The right behaviour has to be the lazy behaviour, or it will not happen — if the easy command is the safe one, nobody has to remember.

What runs must be what you reviewed

The software that actually executes should be provably the software you reviewed — built and placed by a trusted process, not swapped in afterward. This connects directly to proving the bytes that run: verify the deployed artifact against the reviewed source by content hash, so that a divergence between "what we reviewed" and "what is running" is caught rather than assumed away.

A signed, versioned registry for the agent's own tools

The agent's tools and skills are themselves a supply chain — and since a skill can carry a permission verb, a swapped or shadowed tool is a privilege change. Keep a signed registry of the tools the agent may use, with a supersession record so that an old, withdrawn version cannot be replayed in place of the current one. A tool that looks like the real one but is not is exactly what signing is for.

Hardening a system that already exists

Hardening is a different job from building, and in some ways a harder one. You did not choose the structure, the structure is often wrong, and — the part people underestimate — you have to find out what the system actually does before you can change it safely. A running system is full of gaps between what the code says, what the config claims, and what is really executing.

The order here reflects that:

1. Map the running system — establish what is actually running, as opposed to what the repository or the documentation says.
2. Find the class, not the bug — the highest-leverage move in the whole effort: name the defect class and enumerate every sibling, rather than patching the one instance you found.
3. Prove the bytes that run — verify against what the machine serves, not against the working copy.
4. The privileged-path trust class — a worked example of the class method, and the most common escalation route in a system an agent touches.
5. Reproduce from zero — history hides defects; a build you cannot reproduce on a clean machine is a build you are trusting history for.

Throughout, the review discipline in security-reviews/ applies: inspecting a live system can break it, and reading a live service's config can disclose a secret. Read those before you touch a production machine.

The one structural move still comes first

Even when hardening, the first real move is often the structural split from north star 2. If the secrets and the attack surface still live in one process, patching that process will not hold. Sometimes the honest hardening plan is: separate the trust boundary first, then harden the pieces.

1. Map the running system

Before changing anything, establish what is actually running. The repository, the documentation, and the deployed configuration routinely disagree, and the gap between them is where defects hide. A hardening plan built on the repository's version of reality hardens a system that does not exist.

The readable artifact is not the thing that governs

This is the single most important idea in hardening, and it recurs on every layer:

The artifact you can easily read — the file in the repository, the schema in source, the convention a document cites — is a description. The thing that actually governs behaviour is the installed config the service parsed, the bytes the running process loaded, the live database's actual structure, the file a referenced convention really contains. They are often not the same.

And the describing artifact always wins your attention, not because anyone is careless but because it is cheaper to consult. The repository is open in your editor; the deployed file needs a connection to the machine. The schema file is right there; the live structure needs a query. Under time pressure the honest reviewer reaches for the cheap one and gets a confident, internally consistent, wrong answer. The defect is in the gradient, not the person.

The discipline: before citing any artifact as evidence, say which kind it is. "The repository says X" is a claim about the system; only the running system is the system. A divergence between the two is itself a finding worth having — so read both, and label which is which.

How the running system lies about itself

Concrete ways the live system differs from its readable description, each observed in practice:

• A deployed file replaced by a mount. The file at the deployed path can read as empty or stale because the real content is mounted over it inside the service's namespace. A search of the host path searches a placeholder, and finds nothing — which is indistinguishable from "nothing to find." Establish you are reading the bytes the process loaded (inspect the unit's mounts), then read the source those mounts name.
• A reload that failed while leaving the edited file on disk. An "apply" or "reload" command can fail and leave the old configuration serving while your edited file sits on disk looking applied. Never accept an apply command's exit code as proof the new config is live; prove it with a request whose answer differs between old and new.
• A schema change that did nothing. A "create if not exists" with a new constraint does nothing to a table that already exists, and the system catalog may still report the old structure. Prove a constraint from the live database — or by attempting the write it is supposed to refuse — never from the schema file.
• A guarantee gated on per-connection state. A database constraint that reads as a durable schema guarantee can actually depend on a per-connection setting that defaults off, so it holds on every connection the application opens and silently does nothing on any other. A control that is correct in the harness and absent in production is not a weak control; it is an invisible one.

Find the file that sets a value before you explain it

A value read live is authoritative but unargued — it carries no reasoning. The reasoning you find in a neighbouring file is usually about something else. Before writing up a setting as a defect, find the file that actually sets it (the service unit and all its overrides, the wrapper that launched the process), because that is where a deliberate, documented tradeoff would have been recorded. A documented tradeoff written up as an unrecognized defect wastes everyone's time and misdescribes a colleague's judgement — which is how a correct finding gets argued with instead of fixed. Listing a directory of overrides is not reading them; open them.

2. Find the class, not the bug

The highest-leverage move in a hardening effort. When you find a defect, resist fixing that line. Name the class it belongs to, then enumerate every sibling in the system.

Why the class beats the instance

A single fix closes one hole. Naming the class closes every hole of that shape — including the ones no text search would ever have matched, and the ones that do not exist yet but would be introduced by the next change. A patch to one instance leaves the author believing the problem is solved while its siblings sit untouched.

The worked example this playbook keeps returning to is the privileged-path trust class (its own section follows): the discovery that a privileged process trusted a file a lesser account could write was not fixed by patching that one call. It was fixed by naming the class — a privileged mechanism must not trust a path a lesser account controls — and then finding every sibling: the config sourced as root, the privileged task whose target sat in a writable directory, the ownership change applied to a resolved symlink, and more. More than a dozen siblings in one case, several of which were invisible to any search.

Ask the property, not the list of forms

The method that finds the class is to ask the underlying property, never to check against a list of known-bad forms:

• Not "is this one of the flags I know puts a secret in the command line" but "does the value reach the command line?"
• Not "is this one of the five privileged calls I found by searching" but "does a privileged process trust a path a lesser account can write?"

A checklist of forms always has a hole shaped exactly like the form nobody listed. The property has no such hole. Phrase the defect as a property, and the siblings reveal themselves.

Observation beats enumeration

A text search finds the forms you thought of. It cannot find the ones you did not, and it cannot tell code from a comment that quotes code. For a property about what a privileged process actually touches, do not enumerate statically — observe the real operation.

Trace the real privileged operation (the actual deploy, the actual install) and judge every privileged event by the ownership of what it touched: every program executed, every file handed to a privileged interpreter, every path written. Observation reproduces every sibling you found statically and turns up the ones static analysis modelled away — because it tells you what did happen, not what could happen in the forms you anticipated. (Trace paths and process names, never argument values, so the trace does not leak a secret while hunting one.)

A corollary: a fix that needs exceptions is a sign the structure is wrong

When your class-level check passes everywhere except a handful of entries that need exceptions, treat the exceptions as a signal — not that the check needs tuning, but that the structure has a flaw the check is papering over. The strongest fix dissolves the class entirely rather than checking for it: arrange things so the dangerous condition cannot arise, and the check then passes with no exceptions at all. A rule that dissolves the class beats a check at any layer.

3. Prove the bytes that run

A proof that reads the developer's working copy proves the wrong bytes. When hardening, the gap between "what we reviewed" and "what is executing" is not hypothetical — it is where the defects live. Every proof must verify against what the machine actually serves.

Verify against a trusted source of the running artifact

Do not read from a developer's checkout, which any lesser account on a shared machine might have influenced. Build the comparison from a trusted copy of the artifact at the exact version the machine is running, and compare it byte-for-byte against what the host actually serves. The deploy process's own working copy is a build input, never the thing you trust — identify the running version by content hash and verify against that.

Three-way separation when a toolchain is involved

When verifying something that needs a toolchain to build (a compiler, a transpiler), separate the cases so a trusted result is not contaminated by an untrusted one:

• The fully trusted case: built by the privileged account, from the trusted source, with none of the developer's installed tooling present.
• The partially trusted case: built by the privileged account with a single, pinned, privileged-account-held copy of the one tool it genuinely needs.
• The reported case: built by the developer's account — labelled as reported, because an account cannot prove its own integrity, and counted as a claim rather than as proof.

Inside such a proof, turn the developer's tool directory into a trap: make each entry record any privileged invocation and run nothing, and make any secret file a synthetic decoy. If the real tool or the real secret is ever touched, the proof goes red by construction.

The anomalous state is a free sensitivity test

A proof that has passed every time it has ever run cannot honestly be called capable of failing — you have not seen it go red. So when you find the system in an unusual but harmless state, run your instruments against it before you tidy it away. A provenance check that asserts "the running process is newer than the newest source file" is only proven to work when it finally meets a staged-but-not-restarted machine and goes red. The odd state the world handed you is a sensitivity proof; normalizing it to get a clean reading throws it away.

Enumerate the governing set, not the convenient subset

When a proof asserts something about "the files that govern behaviour," enumerate the whole governing set — not the subset that happens to be easy to match. A freshness check that globs one file extension misses a governing file of a different extension that is also read at startup. Derive the set from what the system loads, and the proof covers the claim instead of a convenient slice of it.

4. The privileged-path trust class

A worked example of finding the class, not the bug, and the most common privilege-escalation route in any system where a privileged process and a less-privileged account share a machine — which is exactly the situation an agent with real access creates. This one class is worth internalizing because its siblings are everywhere and most of them do not look dangerous.

The class

A privileged process must not execute, source, read-into-an-interpreter, run version control with trust against, or change the ownership/permissions of any path that a less-privileged account can write — or can replace with a symbolic link.

If a lesser account can write the file, or swap a parent directory for a link, then the "privileged" operation is really running the lesser account's content with elevated authority.

The siblings, several of which no search would match

Instances found by naming the class and enumerating, rather than by searching for a pattern:

• A privileged deploy step sourcing a developer-account configuration file. Sourcing executes it, and an exported variable (a path, a preload, an interpreter option) then steers every later privileged command in the script.
• Version control run as root inside a repository a lesser account owns. Hooks and configuration in a repository the attacker controls run with the privileged account's authority.
• An environment value inherited from a developer file into a privileged service's environment, then used to locate an executable after only a weak existence check — turning "can write a config file" into "can choose what a privileged service executes."
• Ownership and permission changes that follow symlinks. Standard tools differ in whether they follow links; a redundant permission change on a path a lesser account can replace with a link to a sensitive system file hands that file's ownership away.

The lesson is that the dangerous ones do not look like privilege escalations. They look like ordinary deploy plumbing.

The fix patterns — reuse these

• Write as the lesser account, not as root. Have the privileged process hand bytes to the lesser account to write, or drop privilege to write; never touch the lesser-owned path as root.
• Materialize from a trusted, privileged-held source at the exact version, into a privileged-owned temporary location, and operate only on that.
• Use no-follow operations when a privileged process genuinely must touch a path — open, inspect, and change it without following symlinks — never a bare operation on a path a lesser account can swap.
• Refuse a non-privileged-held target at the consumer. The component that runs or reads the path refuses it unless the leaf and every parent is privileged-held — and this refusal is proven by watching it go red against a planted bad path first.

Make it a check, because it is mechanically checkable

This class is enforceable: for every privileged reference — a scheduled-task target, a service's executable path, a startup-read config, a writable-path grant — assert the leaf and every parent is privileged-owned and not group- or world-writable, or the installer refuses. The platform usually cannot check this for you, so the installer must. And re-ask the question from scratch after closing any one route: a mitigation that merely changes which privileged mechanism is exposed, without changing who can write it, will pass every test aimed at the door you just locked while leaving another open.

5. Reproduce from zero

A build you cannot reproduce on a clean machine is a build you are trusting history for. The accumulated state of a long-lived machine hides defects: a thing that works only because of some manual step taken months ago, by someone who is gone, that nobody wrote down.

The discipline

• Build a fresh machine from the documented procedure alone — no manual steps, no "oh, you also have to..." If the documented road does not produce a working system, the road is the deliverable that is broken, and that is itself the finding.
• Snapshot it clean, roll it back, and build again from zero by the one documented command, to prove the procedure is actually repeatable and not quietly dependent on leftover state.
• Run your full verification on both the aged machine and the fresh one, and demand they agree — zero differing results. Every difference is a real defect: something the aged machine had by accident that the documented procedure does not provide, or something the fresh machine acquired that should not be there.

Why the differences are always worth chasing

In practice, the verdicts that differ between an aged machine and a fresh one are never noise. Each one is a thing the old machine was getting for free from its history:

• A capability that worked on the aged machine because a directory happened to already exist, and failed on the fresh one because the documented procedure never created it — invisible on the old box, fatal on a new customer's.
• A dependency pulled in as a recommendation of something else, present on one machine and not the other.
• A paused or disabled state that silently persisted, doing real work nobody had recorded as a control.

Close every difference; do not explain it away. A difference that is genuinely per-machine (a real, legitimate per-host sign-in state, say) is asserted at a level where both machines read the same verdict, with the host-specific fact named explicitly.

The aged machine is the alibi; the fresh machine is the test

The temptation is to trust the machine that has been running fine for months. But "running fine" on an aged machine proves only that its accumulated state is self-consistent, not that the system is sound or reproducible. The fresh machine is the one that tells you the truth, because it has no history to lean on. When the first real customer provision happens, it will be a fresh machine — so the fresh machine is the production case, and the aged one is the alibi.

Security reviews

Hardening is not finished when the controls are in place. It is finished when an independent review has tried to break them, each finding has been reproduced on the real system, and the result is recorded honestly — including the parts that could not be verified and the guarantees that genuinely do not hold.

This section is about doing that review well. It has two halves that are easy to get wrong:

• Not breaking the thing you inspect. A security review has a long history of burning the system it was meant to assess, by commands that do not look dangerous. See review discipline and inspect by property, not content.
• Not fooling yourself. A check that passes can be answering the wrong question, testing the wrong data, or structurally incapable of failing. See red-first proofs and the scorecard and what "done" means.

Contents:

1. Review discipline: don't burn what you inspect
2. Inspect by property, not content
3. Red-first proofs and the scorecard
4. The adversarial pass
5. Probing production safely
6. What "done" means
7. A review brief template

The recurring ways a check can be confidently wrong are collected in reference/instrument-failure-patterns.md — worth reading alongside this section, because most of a reviewer's mistakes are one of a dozen shapes.

1. Review discipline: don't burn what you inspect

A security review must not break the thing it inspects. This sounds obvious and is violated constantly, because the commands that do the damage do not look dangerous.

Read-only intent is not read-only effect

The first reflex to unlearn: "I'm only looking at it" does not mean the command only reads. Many ordinary inspection commands write:

• A version-control status check can refresh an index — a write — and if run as the wrong user in a tree another account owns, it can leave files that block that account's own work.
• The client of a socket-activated service can start the service you meant to inspect, just by connecting to ask about it.
• Lock files, cache warms, and state files are all reads that write.

Before inspecting anything, ask what your inspection writes. If you cannot name the genuinely read-only path, you have not found it yet — and often the real read-only path is just listing a directory rather than invoking a tool.

Check what account your remote shell is

A single "log in as the privileged account" line in your connection configuration silently promotes every casual one-liner you run for the whole session. Check the effective remote user before you reach for a "harmless" command, and never run version control (reads included) as the privileged account in a tree a lesser account owns.

Verbs that read like inspection and are actually mutation

Some commands are named like reads and are writes. A "show" command can re-expose something that was hidden. A status command may not have a status verb at all, so your "status" argument becomes the name of a thing it acts on. A "reset the confused state" reflex can cause an outage. The rule: confirm the command is read-only against its actual documentation, not against what its name suggests.

Tightening a permission is a mutation, and it can break the reader

Hardening a file's permissions during a review is a change, and it can stop the service that reads the file from reading it. If you tighten a file a service depends on, hand it to the service's group rather than locking it to the owner alone, and prove the service still reads it afterward. An in-place edit can also silently change a file's owner or mode as a side effect of how the edit tool writes; re-assert and verify owner and mode after any such edit.

Sanitize the whole environment, not the one variable that bit you

When a leaked environment variable causes a problem, the fix is to start from an empty environment with an explicit minimal set, not to unset the single culprit. Unsetting the one that bit you guards the incident; starting clean guards the property, and the next leaked variable is one you have not met yet. Beware, too, that a scratch test environment is often cleaner than production — so a test can pass for the very reason production fails. Make the test set the dirty condition on purpose.

2. Inspect by property, not content

A security review's output is read by people, stored in logs, and — when an agent performs the review — sent to a model provider and held in its context. So a review must never print a secret, not even to confirm a fix worked, not even once.

The one rule

Inspect by name and by property. Never by content.

Prove a secret exists (test for readability), prove its length, prove its permissions, prove a key name is present. Never print the value.

The commands that disclose a secret without looking like it

The dangerous commands are not the ones that obviously handle secrets. They are ordinary inspection commands aimed at a file that happens to contain one:

• Reading a live service's configuration, environment, unit definition, or process environment is a credential-disclosure operation, and neither a "cat" nor a "grep" announces it. Assume a secret is in any such file before you read it.
• A broad substring filter is how this fires: searching a process environment for a service name matches the service's secret and prints it. Filter to the specific keys you want, by anchored name, never a broad substring.
• Extract key names only when you need the shape; read which secret-store item a config references, not the value it holds.

The forms that defeat a naive filter

Three shapes bypass "I filtered, so I'm safe," and each is worth recognizing:

• A directive that carries its value inline. In many config languages a directive name and its secret argument share a line by construction, so filtering to the directive is filtering to the secret. A directive-name filter is a broad read wearing a narrow costume.
• A secret that is the key, not the value. A redactor that masks fields whose name looks secret will happily print a store shaped as { "<secret>": {...} }, because the secret is the key and the redactor only looked at values. Before printing any parsed structure, ask where in the shape the secret sits — key, value, array element, or inside a URL.
• A credential inside a URL. scheme://user:password@host is a credential that matches no password-field filter. It is the form that slips past every check because it does not look like one.

The lesson under all three: a filter encodes an assumption about where the secret is, and the assumption is exactly the hole. Prefer a reader that cannot print a secret by construction — if the store has a provisioning tool, that tool has a listing mode built not to leak — over a hand-written redactor you are trusting to catch every shape.

When you need the shape, project, don't redact

To understand a file's structure without risking its content, project the first token of each line rather than redacting what looks secret. A redactor is a denylist — it emits everything except what its pattern anticipated, so an unusual-looking secret walks through. Projecting only the first field is an allowlist — it emits only what you asked for and discards the rest, so a secret that does not look like one cannot survive it. Use a redactor to read prose or logic safely; use first-token projection to read structure.

And never use a redactor to compare two files. A redactor is lossy by design, so piping a comparison through it makes differences vanish exactly where the content is densest — and it fails toward "same," which is the reassuring direction. To compare, hash the originals and emit only the verdict; a boolean leaks nothing.

If you leak one anyway

Treat it as compromised and rotate it. Prove the rotation by the old secret being refused — reading the reason, not just the status, so you are not fooled by an expiry that would have happened regardless. And record the exposure on the credential item itself, so the next person to handle it can see it happened. A disclosure recorded only in a transcript dies with the transcript.

3. Red-first proofs and the scorecard

A review's credibility rests on whether its checks can actually fail. This section is the review-time application of prove, don't promise, plus how to report the result honestly.

Every proof is watched failing first

For each property, name the input that should make the check fail, feed it that input, and watch it go red before you trust the green. A check you have only ever seen pass is a decoration. This applies to the proofs themselves as much as to the system — in a thorough review, the proof is often the weak part before the code is: a filter that blocked only one form of an attack, a control that never actually ran, a positive control that was invisible. Each of those is caught only by holding the proof to the red-first standard.

The proof must be able to say "I cannot tell from here"

Beyond pass, fail, and absent, a check needs a fourth state: unverifiable from this vantage. A check running on a machine cannot see a control that protects the machine from outside. Rather than claim a pass it cannot support or an alarm it cannot justify, it states what it can see, names the control it cannot, says who could check, and refuses the conclusion. Critically, this state does not trip the alarm — an honest "I cannot judge this" treated as a failure trains people to ignore the failures that matter.

The scorecard: generated, grouped, honest states

Collect the proofs into a generated scorecard — run on the real system, grouped by the control each proves — with states that never collapse into one another:

• HELD — the proof ran and the property holds.
• BROKEN — the proof ran and the property is false. A known, accepted red is annotated with its closure condition and never excused — it stays visibly broken until it is actually fixed.
• UNVERIFIABLE — the proof cannot judge from this vantage. Never collapsed into HELD.
• REPORTED — held as reported by a component that cannot be fully trusted to prove its own integrity. Labelled as such, not counted as proven.
• OPEN — a property named but not yet given a proof.

The states matter because collapsing them is how a review lies politely. An unverifiable thing reported as held is a false pass; a broken thing reported as "known issue, moving on" is a hole with a label on it.

The measuring instrument is itself a surface that lies

The scorecard is code, so it gets the same scrutiny as the thing it measures. Ask what the counter rounds, what a summary line truncates, what a clean exit could be hiding. In practice a scorecard will find its own bugs — a category of failing proof miscounted as a decline, a note that printed only the last failing line so earlier failures were invisible. Hold the instrument to red-first too: feed it a known failure and confirm it reports it faithfully, with a persistent record it cannot be made to overwrite.

Reading a result correctly

• The status an operation returns through a pipeline may belong to a different command than the one you are judging. Read the status of the command you mean, not of whatever ran last in the chain — or a check that cannot fail will cheerfully certify anything.
• Reading the reason for a failure, not just its category, is what separates a proof from a coincidence. A refusal, a timeout, and an unreachable host are different causes that can share a status; only one is usually your claim.
• A pass needs adversarial scrutiny more than a fail does. A failing check announces itself; a passing one that is quietly answering the wrong question does not. Check hardest when your conclusion is comfortable.

4. The adversarial pass

After the controls are built and the scorecard is green, an independent reviewer attacks the system — given the security claims and nothing of the build's history — from the position of a compromised internal component.

Give the adversary the claims, not the map

The reviewer should be briefed with what the system promises and the vantage of a realistic attacker (for an agent system, a foothold as one of the lesser accounts) — and not with the build's internal history, decision log, or architecture tour.

The reason is practical, not ceremonial. A reviewer who is handed the full history spends its budget re-deriving what you already know and runs out before finding anything new. A reviewer who has only the promises spends its budget testing the promises — which is the entire point. Give it the claims and the foothold; let it discover the structure by attacking it.

Every finding is a claim until reproduced

A finding the reviewer hands back is a claim, not a fact. Reproduce it on the real system before it counts. This cuts both ways: it filters out false alarms, and — just as important — it means a reviewer's "it holds" is only trustworthy if "it holds" was an acceptable answer in the first place. A review brief that implicitly demands findings manufactures them; state explicitly that "this control holds" is a valid and welcome result.

Record the honest result, including what does not hold

The outcome of an adversarial pass is rarely "everything holds" or "everything is broken." It is usually "holds on most, with named exceptions." When a guarantee genuinely cannot be enforced at a given layer — a behavioural property that the platform cannot make kernel-level, say — record that honestly: name the layer where it does hold (code, architecture, process), name the residual that remains, and print it rather than papering over it. Overstating a guarantee you cannot enforce is worse than naming its limit, because it is the limit nobody knew about that gets exploited.

This is a gate, not a formality

For a system running stranger-adjacent or untrusted-input-adjacent code, an adversarial review is a gate that runs before the first external user, not a box ticked afterward. The whole point of assume-compromise is undermined if the one exercise that tests the assumption happens after real data is already at risk.

Separate the behaviour you observed from the motive you inferred

When the reviewer (or anyone) escalates a finding, keep two things apart: what was observed is evidence; why someone did it is inference. Report the behaviour flatly and mark the motive as a guess, or leave it out. A finding that rests on an inferred motive is fragile — the motive may be wrong, and when it is, a correct technical finding gets argued with instead of fixed. An argument from a fact about the mechanism survives; an argument from someone's presumed intent decays the first time the presumption is wrong.

5. Probing production safely

Testing a live system against real behaviour is necessary — a config read proves nothing — but probing production can change it, cost money, or deepen the very problem you are measuring. A few disciplines keep the probe honest without harming the system.

A status code is a bucket, not an answer

The same status can come back for several different reasons. A removed resource, a resource that never existed, and a real resource can all return the same redirect on one host. A probe whose only output is a status code means almost nothing on its own.

Always run a known-negative and a known-positive control alongside the probe.

The negative control calibrates what "should fail" looks like; the positive control proves your instrument can detect a success at all. Without both, a green result might mean the control works — or might mean your probe is blind.

An empty result is not a clean result

A permissive access rule returns nothing when there is nothing to return — so an authorization probe over an empty data store passes, and leaks the day the store fills. This is a time bomb whose fuse is the product succeeding, and you cannot find it by re-reading the check, because the check is correct. The fix is one assertion: a known-present positive control in the same run — a record the same probe sees in the same response when authorized. If the canary is invisible, the run is void, not passing. Ask this especially when the store is nearly empty, which is exactly the pre-launch window in which reviews happen.

The access pattern is part of the test

Exercising a control in one access pattern proves it for that pattern only. A rate limiter verified by a sequential series of attempts tests its arithmetic, not its behaviour under a concurrent burst — and a concurrency control exercised serially can pass completely while failing wide open under the burst an attacker gets for free. Before calling any control proven, ask which access pattern you exercised and which one an attacker gets for nothing: concurrency, ordering, repetition, a partial or aborted request, a second identity, a cold cache. And when you test concurrency, create a real race — release the participants from a shared barrier so they are all running at the same instant — because a burst assembled by a loop runs sequentially and proves nothing.

Know what is actually bounding the damage

Beware a limit that is really a performance artifact. If the damage plateaus not because a control bound it but because a slow operation or a serialized write did, then a faster machine would raise the ceiling with no code change and no alarm — improving performance would weaken security. If you cannot name the control doing the bounding, there is no control; there is a coincidence that a routine optimization will remove.

The instrument is made of the thing you are measuring

Diagnosing a rate limit costs requests against that limit. Probing a fragile service can tip it over. Plan your probes before running them, and account for the cost of the measurement as part of the measurement. And verify a host's external protections from the outside — a host cannot see the provider firewall protecting it, so from inside, protected and exposed look identical.

When the harness says the subject is broken, check the harness first

A broken test harness reports the subject as defective, and the reflex is to believe it, because a failing check feels like diligence working. An impossible reading — zero listeners on a port you are connected over, zero rows in a table you just wrote — is a harness fault, not a finding. Give the harness its own positive control, and suspect the instrument before you "fix" a defect that may never have been there. A false alarm acted on is pure damage: the repair to a problem that did not exist, or worse, the deletion of a sound control for being "vacuous."

6. What "done" means

A review's hardest job is resisting the many ways a system can look finished while being broken. "Done" has a precise meaning here, and it is stricter than most teams assume.

Done means the real job was exercised once, end to end, and watched

Not "the services are enabled." Not "the perimeter is clean." Not "every check is green." The actual thing the system exists to do, performed once, from one end to the other, and observed.

The failure this rule is built from: a system fully provisioned, passing every verification check, while the one path that mattered had been dead for an hour — because the checks asserted existence and configuration, never once exercising the real path. "It parses," "the service is active," "the path resolves" are not this. You must run the real path and watch it work.

A check that has never gone red is not a check

Repeated here because it is the single most common way a review certifies nothing. Before trusting any check, name the input that makes it fail and prove it fails on that input. A green that has never been shown to go red is precisely what lets a broken system sit looking healthy.

In a shared tree, done means merged and re-proven there

Checks passing on a branch prove the branch, and nobody runs a branch. A result from an unmerged branch is work, not done. Re-prove on the integration line, against the artifact that actually ships.

The counterfactual

Before reporting a negative result as proof, ask: would I have seen this exact result if the change had never been made? If yes, you have a result, not evidence. Run a control that separates the cause you claim from the causes that would have produced the same result anyway, and say which test proved which. Reading the reason for a result — not just its category — is what upgrades a guess to a proof.

The disk is what ships, not the version-control pointer

Deploy tooling and servers read files, not repository objects. A working tree can disagree with the committed pointer, and the wrong content can be one command from shipping while every content check passes by luck. The assertion that closes this regardless of content: every file in the deployable set matches what was reviewed, by hash. Freeze a shared tree before measuring it for a decision; a measurement of a tree five people are writing to has a shelf life shorter than your report.

Recognizing the fakes on sight

A compressed index of the shapes that fake "done," each worth recognizing:

• A liveness check that only proves a file exists.
• A fallback quietly doing the primary job, so the system looks up while producing wrong output at full confidence. A fallback that fires always is indistinguishable from success unless it is made loud or gated.
• A check whose subject and the claim's subject are different nouns — proving a login works against the old database after a migration, for instance. Name the artifact under test, not just the behaviour.
• A control placed at the wrong layer, running after the thing it was meant to prevent has already been granted — so it can only decline to use a capability, not remove it.
• A safety property that holds today by an accident nobody chose — an ownership that happens to be set a certain way, a service that happens not to run. It passes every test you can write, because the world is simply about to change under it. The test for this one is a question, not a check: what would have to change for this to stop being true, and is that change already planned?

Several of these have their own worked treatment in reference/instrument-failure-patterns.md.

7. A review brief template

How you write the brief for a review largely determines its quality — whether the reviewer is a person or an agent. A reviewer handed a juicy target and a vague warning will read the secret, rediscover what you already knew, and manufacture findings to seem productive. The brief below is structured to prevent each of those.

Put the safety rules first, above the scope, in the imperative. An agent that reaches the interesting part before it reaches the warning has already done the damage.

Template

Safety rules (read first, non-negotiable)

• This is READ-ONLY. State exactly what read-only excludes for this target: no restart, no reconfigure, no write, no permission change, no credential rotation.
• Forbidden commands. Name the specific commands that must not be run against this target — the destructive ones, the ones that look like reads but mutate, the "reset" reflexes. Do not rely on "be careful."
• Never print a secret value. Inspect by name and property, never by content. Treat any config, environment, unit, or process environment as containing a secret until proven otherwise.
• Assume your inspection can write. Name the read-only path; if you cannot, do not run the command.

Scope

• The exact systems, services, and surfaces in scope — and, explicitly, what is out of scope.
• The vantage point: for an adversarial pass, the foothold the reviewer is assumed to have.

What we already know

• The known root cause or the known-good behaviour, so the reviewer reviews rather than rediscovers. A review that re-derives what you already know is a review that ran out of budget before finding anything new.
• For an adversarial pass, invert this: give the claims and withhold the history, so the reviewer tests the promises instead of re-deriving the design. (The two brief styles are different; choose deliberately — see the adversarial pass.)

What a good result looks like

• "It holds" is a valid and welcome result. Say so explicitly, or you will get manufactured findings.
• Every finding must distinguish confirmed (reproduced on the real system) from inferred, and show the evidence command (sanitized of any secret).
• A finding's observed behaviour is evidence; its presumed motive is inference — report the first flatly, mark the second as a guess or omit it.

Deliverable

• A list of findings, each with: the property it violates, confirmed-vs-inferred, the evidence, and the blast radius if real.
• The honest state of each claim checked, including anything unverifiable from the reviewer's vantage and any guarantee that genuinely does not hold at the layer it was claimed.

The single highest-value line in any review brief is "say 'it holds' if it holds, and show me how you tried to break it." It converts the exercise from theater into measurement.

Reference

Supporting material for the playbooks — read when you need a definition, a fuller treatment of a recurring pattern, or pointers to the public literature.

• Glossary — the terms this playbook uses, defined plainly.
• The named threats — a threat catalogue for agent systems, with an honest note on which threats the controls in this playbook answer and which they do not.
• Instrument failure patterns — the recurring ways a check can be confidently, articulately wrong. Most reviewer mistakes are one of these shapes; recognizing them on sight is worth more than any single fix.
• The artifact that governs — the family of bugs that come from reading a description of the system instead of the system, across four layers.
• Further reading — public standards and literature for deeper study and for grounding the reasoning here in established practice.

Glossary

Terms as this playbook uses them. Where an industry term has a looser common meaning, the definition here is the narrower one the playbook relies on.

Agent. A system driven by a language model that takes actions — reads inputs, calls tools, writes files, runs code — in pursuit of a goal stated in natural language. The source of both the capability and the risk.

Append-only record. An audit log to which entries can be added but never silently changed or removed. Even a correction is a new entry. See the off-box record.

Assume-compromise. The design stance of treating the agent as already under an attacker's influence, and ensuring safety holds anyway. See north star 1.

Blast radius. How much an attacker can reach or damage given a particular foothold. Least authority and isolation are about shrinking it.

Broker. The single-purpose, privileged component that holds credentials, performs privileged operations, and writes the record — accepting only structured requests through one entrance, and exposing no operation that returns a secret. See the credential broker.

Capability. What a component is able to do, as opposed to what it is told to do. Controls based on capability survive a successful injection; controls based on instruction do not. See capability over instruction.

Counterfactual (in verification). The question would I have seen this exact result if the change had never been made? If yes, the result is not evidence of the change.

Egress control. A default-deny outbound policy permitting only an allowlist of specific destinations, enforced in the kernel below the application. See egress control.

Human-in-the-loop (HITL). The requirement that a person explicitly authorize any outward or irreversible action, through a confirmation the agent cannot reach or forge.

Injection (prompt injection). An attack delivered as content the agent reads, steering it to act against its owner. Defined out of existence at the boundary by choosing capability from channel and identity, never from content.

Least authority. Granting each component exactly the access its one job needs and nothing adjacent. See least authority.

Prove-don't-promise / red-first. The rule that every protection has an executable check watched failing before it is trusted to pass, verified on the real system by behaviour.

Sandbox. The disposable, networkless, unprivileged environment where code the agent writes actually runs — built so the agent's freedom inside it cannot reach anything that matters. See sandboxing untrusted code.

Scorecard. A generated, control-grouped report of every proof, run on the real system, with honest states (held / broken / unverifiable / reported / open) that never collapse into one another. See the scorecard.

Shared fate. The property that compromising one thing compromises another. The goal is to remove it wherever a single breach would otherwise cascade — most sharply, by keeping the audit record off the machine it records.

Trust boundary. The line between a component that is trusted and one that is not. The first structural move is to put one between the agent (untrusted) and the broker (trusted). See split the trust boundary.

Trust class (of defect). A family of defects sharing a property, as opposed to a single instance. Hardening finds the class and enumerates every sibling rather than patching the one bug found. See find the class.

Typed verb. A structured, schema-validated request for a boundary-crossing action — never a command string — which the broker validates and performs. The only way anything consequential leaves the system.

Unverifiable (fourth state). A check's honest report that a claim depends on something outside its vantage point, so it refuses the conclusion rather than faking pass or fail.

The named threats

A control list cannot tell you what it misses; a threat list can. This catalogue names the threats an agent system faces, maps each to the control that answers it, and — the part most write-ups skip — states honestly which threats the controls in this playbook do not answer.

The categories follow the shape of published work on agentic-application security (see further reading); the mapping and the honesty about gaps are the useful part.

The threats, and how well they are answered

Memory / knowledge poisoning. For a system that persists what it learns — ingesting content and distilling it into standing knowledge the agent later trusts — a corrupted corpus can change what the agent believes through the product's intended path, with no exploit. It is a threat a product creates for itself by adding a learning feature, which is why a generic threat list will not flag it: only modelling your own system will. Out of scope for this playbook. The controls here cover the agent / broker / sandbox / record / egress architecture, not the hardening of a persistent memory or knowledge store — that is a distinct discipline, and this playbook does not claim a method for it. A system that builds such a feature should treat it as its own design problem rather than assume the controls here extend to cover it.

Tool misuse / tool shadowing. A tool that impersonates a real one, or an old withdrawn version replayed in place of the current one. Answered by a signed, versioned tool/skill registry with a supersession record. See the supply chain.

Privilege compromise. A component gaining authority it should not have. Answered by account separation, the broker, and the system/content separation that keeps the agent reading untrusted content from ever holding privileged capability. See least authority and the privileged-path trust class.

Intent-breaking / goal manipulation. Steering the agent's objective rather than a single action. Partly answered. The structural rule — the agent reading untrusted content never holds the dangerous capability — limits the damage, and the small verb set limits the reach. But nothing detects that a goal has been changed; only that actions are bounded. That is the honest limit, and it is an argument for observability rather than for another gate.

Misaligned or deceptive behaviour. The agent acting against its owner while appearing compliant. Largely unanswered by technical controls. The real control here is attribution and audit plus a human on anything outward. Say this plainly to anyone evaluating the system rather than claiming a defence you do not have.

Rogue agents. An agent operating outside its registration or identity. Preconditioned on identity. An agent that cannot be reliably identified cannot be found rogue, so robust agent identity is the prerequisite; the off-box record is the process half that makes a rogue action visible after the fact.

What this honesty buys

Stating plainly that some threats have no technical answer here, that one is out of this playbook's scope, and that others are only partial, is far more useful than a reassuring list of controls — to a buyer, to a reviewer, and to the next engineer deciding what to build next. It tells you where the residual risk is, which is exactly what a control list hides. And it carries a general lesson for anyone adapting this material: a threat that your own product's shape creates will not appear on a borrowed checklist, so the controls you need may be ones no generic list — this one included — contains.

Instrument failure patterns

The recurring ways a check can be confidently, articulately wrong. These are collected separately because most reviewer mistakes are one of these shapes, and recognizing the shape on sight is worth more than catching any single instance. Each has been observed in real work, more than once, by careful people.

A unifying observation runs through all of them: your probes fail green. A probe that errors, matches nothing, or mis-parses tends to produce empty or zero — and almost every check is written so that empty or zero means "nothing wrong." The failure mode of the instrument and the encoding of "all clear" are the same value. So a clean result from a first-draft probe is the reading to distrust, and the discipline is always the same: make it go red on purpose before you believe its green.

The patterns

A check that has never gone red. Green from birth proves nothing — it may be testing the right thing or be structurally incapable of failing, and you cannot tell which from a passing run. Name the input that should fail it, feed it, watch it go red.

The status read through the wrong command. The status you read through a pipeline, an and-chain, or a captured substitution can belong to a different command than the one you mean to judge — so the check certifies anything, including the very absence it exists to catch. The defect is manufactured by the reader, not the author, so reviewing the check's source never reveals it.

A pattern too loose and a check too narrow are the same defect. Both return a clean, confident, wrong answer, because both fail to express the question. A search narrower than the real name misses a live thing; a search broader than the real thing matches everything. Before trusting a tidy result, ask what else it matches and what it misses — a tidy result is the warning sign, not the reassurance.

A search over source answers "does this string occur," never "is this live." Comments, docstrings, string literals, and documentation all match — and the better the documentation discipline, the more broken code lives in comments explaining why it was broken. Anchor to structure, prefer enumerating what a real line looks like over excluding the comment shapes you happened to think of, and to answer "is this live" read the code or observe the behaviour.

An empty result read as a clean result. An empty file looks exactly like a clean one; an absent match looks exactly like a passing check; a deployed path that is really a mount target reads empty from the host. Demand a positive control that must come back non-empty, or the run is void rather than passing.

A negative asserted without a positive control. "This value appears in no response" is worth nothing unless the same run can also show the value appearing when it should. An authorization sweep run entirely unauthenticated samples only bodies that were never going to carry the value. Demand a positive observation, not only an absence.

A detector tested only for false positives. A false alarm announces itself; a silent miss never does — so the cheap test ("will it fire when it shouldn't?") gets run and the expensive one ("can it stay quiet when it should fire?") gets skipped. Ask the cheapest way for the detector to stay silent, build that case, and watch it fire. Beware a signal that is a proxy for the thing (a status code standing in for a body, a handle standing in for an identity) — the day the proxy moves with the thing it was meant to distinguish, the detector goes quiet with no red anywhere.

Red-first validates sensitivity, not correctness. Watching a check go red proves it detects something; it does not prove it asserts the right thing. A check can pass its own red-proof and still defend a regression, if it was built to assert the wrong requirement. Trace the property to the requirement separately — quote the requirement next to the assertion — and treat an articulate, well-argued conclusion as more in need of an independent test, not less.

A control at the wrong layer. A check that runs after the capability it guards has already been granted can only decline to use the capability, not remove it. For any guard, ask what has already happened by the time it runs. The strongest fix dissolves the condition that needed the guard rather than adding a better guard.

A property true by accident. A safety property can hold today for a reason nobody chose — an ownership that happens to be set a certain way, a port something unrelated happens to firewall, a table that happens to be empty. It passes every test you can write, because the world is about to change under it. The test is a question, not a check: what would have to change for this to stop being true, and is that change already planned? Record it as dated-and-accidental with the event that will end it, or make it a real control — never write it down as a control it is not.

A control borrowed from a system you do not operate. A negative assertion ("X is refused") that is true only because someone else keeps it true is not a control — it is a loan, and nothing tells you when it is repossessed. Establish the condition yourself rather than borrowing it, and when auditing, ask of every negative assertion: who operates the thing that makes this true?

A shared premise nobody re-checks. A one-line claim about the code that passes from one report into another's conclusion into a third's decision survives rounds of genuine scrutiny — because each round checks whether the conclusion follows from the premise, and nobody re-checks the premise against the code. The more a premise is reused, the less it is examined, because reuse reads as corroboration. When a claim about the code becomes load-bearing for a decision, read the code again — and enumerate every caller, not the one the claim was written about.

When the harness says the subject is broken, check the harness first. A false red gets acted on — the subject gets "fixed," or a sound control gets deleted as vacuous — so it is more dangerous than a false green, which reality eventually catches. An impossible reading is a harness fault, not a finding.

The artifact that governs

One family of bugs underlies a surprising share of wrong security conclusions: reading a description of the system and mistaking it for the system. It appears on four layers, and it is worth seeing as one shape rather than four tips, because the fix is the same each time — reach the thing that actually governs, and label a description as a description.

The shape

The artifact you can easily read is a description. The thing that actually governs behaviour is something else, harder to reach. They are often not the same, and the divergence between them is itself a finding.

Why the description always wins your attention

Not carelessness — convenience. In every row, the describing artifact is cheaper to consult than the governing one. The source file is open in your editor; the deployed file needs a connection to the machine. The schema is right there; the live structure needs a query and often a credential. The cited convention is a filename you already believe. Under time pressure the honest reviewer reaches for the cheap one and gets a confident, internally consistent, wrong answer. The defect is in the gradient, not the person — which is why discipline, not diligence, is the fix.

The gradient bends your verification too

The same pull that bends your reading bends your verification: you end up checking the claims that are cheap to check and accepting the one that is expensive. An agent (or a person) who verifies four of five claims has not done 80% of the work — they have done the easy 80%, and the unchecked claim is unchecked precisely because it was expensive, which correlates with it being where the truth was hiding. Before reporting, list what you did not check and why; if every omission shares the reason "it was harder," you have measured your own convenience, not the system.

The rule

• Before citing any artifact as evidence, say which column you are in. If you are reading a description, say "the source says X," never "X is configured." A description is a claim about the system; only the governing artifact is the system.
• Do not collapse the two — the description is where intent lives, and a divergence between intent and reality is one of the most valuable findings there is. Read both, and name which is which. Consulting only the governing one loses the drift; consulting only the description loses the truth.

Further reading

Public standards and literature for grounding the reasoning here in established practice, and for going deeper than a playbook can. These are starting points, not an exhaustive bibliography; verify current versions, as several are revised regularly.

Agent- and LLM-specific security

• OWASP Top 10 for Large Language Model Applications — the widely-used catalogue of LLM risks, including prompt injection and insecure output handling. owasp.org
• OWASP work on agentic-AI threats — threat modelling specific to tool-using, multi-step agents, which is where the named-threats catalogue in this repo takes its shape. genai.owasp.org
• NIST AI Risk Management Framework (AI RMF, 100-1) — a governance-level framework for managing AI risk, useful for situating technical controls inside an organizational process. nist.gov

Foundational security principles

• Saltzer & Schroeder, "The Protection of Information in Computer Systems" (1975) — the original statement of least privilege, fail-safe defaults, economy of mechanism, and complete mediation. Every north star here is a descendant of this paper; it remains worth reading in full.
• Capability-based security — the model behind "capability over instruction": authority as an unforgeable token to do a specific thing, rather than an identity checked against a list.
• NIST SP 800-207, Zero Trust Architecture — "never trust, verify explicitly," the formal backing for a trusted network is not a security model. nist.gov
• NIST SP 800-53 — the comprehensive control catalogue; useful as a reference when you need the established name and family for a control you have built.

Isolation and sandboxing

• Linux kernel documentation on namespaces, seccomp, and control groups (cgroups) — the primitives under container isolation and the resource ceilings a sandbox asserts. kernel.org
• systemd sandboxing directives (the execution-environment and resource-control documentation) — the unit-level controls for confining a service: filesystem restrictions, capability bounding, system-call filtering, address restrictions. freedesktop.org
• Rootless-container documentation (the Podman project and the broader OCI ecosystem) — for running disposable, unprivileged, daemonless sandboxes.
• microVM monitors (Firecracker) and application kernels (gVisor) — the stronger-boundary options discussed as the hardening step after disposable containers; read these when you port a settled workload to a kernel-level boundary.

Supply chain

• SLSA (Supply-chain Levels for Software Artifacts) — a framework of increasing guarantees about how an artifact was built and that what runs is what was reviewed. slsa.dev
• Sigstore — practical signing and verification of software artifacts, the mechanism behind "what runs must be what you reviewed." sigstore.dev
• CIS Benchmarks — concrete, consensus hardening baselines for operating systems and common services; a good source for the mundane-but-important host hardening this playbook assumes around its agent-specific controls. cisecurity.org

On verification and honest measurement

The verification discipline in this repo — red-first checks, honest states, the counterfactual, the instrument-failure patterns — is less formally codified in the literature than the architectural material, because much of it is craft. The closest established touchstones are the testing-theory distinction between verification (did we build it right) and validation (did we build the right thing), and the general practice of mutation testing (deliberately breaking the system to confirm the tests notice), which is the formal cousin of a check that has never gone red is not a check.