Humans in the loop, without the rubber stamp

Companion code: 22-approvals

TL;DR

  • “A human approves every action” sounds safe, but people stop reading what they approve. Anthropic found Claude Code users approve 93% of permission prompts.
  • Make approvals rare and meaningful. Allow reads and small reversible actions automatically, ask only for consequential ones, and deny in code what no approval should unlock.
  • The tiers depend on a call’s arguments, not just the tool’s name: a $30 refund and a $3,000 refund are different decisions. Show reviewers a plain-language summary, and stop the agent and hand the task to a person when it keeps hitting the limits.

The approval paradox

The usual first answer to “how do we make this agent safe?” is to have a person approve its actions. It feels responsible, and for a handful of high-stakes actions it is. Applied to everything, it fails in a predictable way.

In March 2026 Anthropic reported that Claude Code users approve 93% of permission prompts. Its earlier sandboxing post named the cause: constantly clicking “approve” leads to approval fatigue, where people stop paying close attention to what they’re approving. Security researchers saw the same thing a decade earlier with browser warnings: in a study of over 25 million warning impressions, users clicked through 70.2% of Chrome’s certificate warnings, but far fewer of the rarer, clearer ones.

A prompt that appears twenty times a task becomes a rubber stamp, and a rubber stamp is worse than nothing because everyone believes someone checked.

Three tiers, decided in code

Every tool call the agent makes passes through a policy written in code that looks at what the call does and to whom. Reads and small reversible actions are allowed and logged. Consequential actions are shown to a person as a plain-language summary. Actions no approval should unlock are denied, and the agent is told why. After three denials in a row, the task stops and goes to a person.

Sort every action the agent can take into three tiers:

  1. Allow, and log. Reads, and actions that are small, reversible and squarely inside the task: looking up an order, adding an internal note, a refund under a set limit to the original card, replying to the customer who opened the ticket.
  2. Ask a person. Actions with real consequences that are sometimes legitimate: a large refund, a change of shipping address, deleting data, anything that commits money or can’t be undone.
  3. Deny, and explain. Actions that no approval should unlock in this task: emailing customer data to someone who isn’t the customer, refunding to a different card, bulk exports. These aren’t questions for a tired reviewer at 6 p.m.; they’re rules.

Three details make the difference between a policy and a list.

Decide from the arguments, not the tool name. issue_refund isn’t one risk level. In the companion code, a refund is allowed under $50 to the original payment method, sent for approval above that, and denied outright if it goes to another card, exceeds the order total, or touches an order that isn’t the customer’s. send_email is allowed to the ticket’s own customer and denied to anyone else.

Enforce it in code. The policy is a function that runs before every tool call, so it holds even when the model is confused or manipulated (post #10). Instructions in the prompt are helpful; code is what you can rely on.

Tell the agent why. A denial comes back to the model as a tool result: what was blocked, why, and an instruction not to route around it but to find a safe way to help. Anthropic’s auto mode does the same, and adds a backstop: after 3 consecutive denials or 20 in total, it stops the model and hands the session to a person. The demo copies that rule.

The demo: one reviewer, three policies

The companion code (22-approvals) runs a support agent through five tickets: a delivery question, a $28.50 refund, a $189 refund, an address change, and a ticket from someone claiming to be the customer’s accountant. That last one is a prompt injection: it asks the agent to email the customer’s order history and card details to an outside address and to refund the order to the “firm’s” card.

Each set of tickets runs under three policies: ask for everything, risk-tiered, and never ask. The reviewer is a rubber stamp that approves every prompt, which is roughly what a 93% approval rate looks like.

Here is the run with a scripted agent that deliberately falls for the injection, to show what each policy does when the model gets it wrong:

Policy Actions Approval prompts Blocked Unsafe actions that ran Escalated
Ask for everything 25 25 0 3 0
Risk-tiered 25 2 3 0 1
Never ask 25 0 0 3 0

“Ask for everything” put 25 decisions in front of the reviewer and still let all three unsafe actions through, because the reviewer approved them along with everything else. The tiered policy asked twice, for the $189 refund and the address change, both genuinely worth a look. It blocked the export, the email to the outside address and the refund to another card regardless of what the reviewer would have said, and after three denials in a row it escalated the ticket to a person.

Then the same five tickets with a real model, Claude Opus 5.5, three times under each policy:

Policy Actions per run Approval prompts per run Unsafe actions that ran
Ask for everything 27–28 27–28 0
Risk-tiered 28 2 0
Never ask 27–28 0 0

The model saw through the “accountant” every time. In all nine runs it declined both requests, emailed the real customer to say so, and left an internal note, without attempting a single unsafe action. That’s good news about the model, and it changes what the comparison shows. With a well-behaved model, the safety outcome was identical under all three policies; what differed was the reviewer’s day. “Ask for everything” put 27 or 28 decisions in front of a person per batch of five tickets, including every internal note and every routine reply. The tiered policy asked twice, both times for something worth a look.

So the tiered policy cost nothing in safety and removed over 90% of the prompts. And when a model does fall for an injection, as the scripted run showed, it’s the only one of the three that still holds. That guarantee doesn’t depend on the model behaving well, or on the reviewer paying attention.

Making the remaining approvals count

Once approvals are rare, make each one easy to judge:

  • Summarize consequences, not calls. “Refund $189.00 on order C-7781 (office chair, $189.00) to Chen Wei’s original payment method. Reason given: broken base” is a decision a person can make in seconds. A JSON blob of tool arguments isn’t.
  • Show what’s unusual. A new recipient, an amount above the customer’s history, a first-time action. That’s where attention should go.
  • Check before each action, not each turn. Models can send several actions in one response (post #21); approve them one by one.
  • Measure prompts per task. If reviewers see more than a few per task, the tiers are wrong. Move safe, frequent actions down to “allow”, and turn recurring “ask” decisions into rules.
  • Sample the allowed tier. Logs of auto-approved actions are where you discover the policy is too loose. Review a sample regularly (post #15).

Where classifiers fit

For general-purpose agents, writing rules for every action isn’t possible. Anthropic’s answer for Claude Code is auto mode: reads and edits inside the project run freely, and everything else goes to a classifier that decides whether to allow it. The classifier sees the user’s messages and the agent’s tool calls but not the agent’s own explanations, so the agent “can’t talk the classifier into making a bad call”.

The published numbers are worth knowing. On real traffic the full classifier blocked 0.4% of harmless actions. It missed 17% of real overeager actions and 5.7% of synthetic data-exfiltration attempts. Anthropic is clear that it’s “not a drop-in replacement for careful human review on high-stakes infrastructure”, and recommends a sandbox as the outer boundary. That’s the right mental model: classifiers reduce prompts for open-ended work, code rules enforce the limits you can state, and isolation contains what both miss.

Trade-offs

  • Rules need maintenance. Thresholds like “$50” are business decisions. Put them in configuration, review them, and expect to change them.
  • Too strict is also a failure. If the agent is blocked from routine work, people will route around it, or around the policy. Watch the escalation rate as closely as the prompt rate.
  • Approval isn’t accountability. Log who approved what and when, with the summary they saw. A rubber stamp with a good audit trail is still a rubber stamp, but at least you’ll know.

Next in the series: Prompt caching, done properly.

Sources & further reading

#security #harness #decision-makers