Result · S1c

The control worked. It cost 28 points.

I tested one safety control on an AI banking agent under prompt-injection attack. It took successful attacks from 50% to zero. It also cut the legitimate tasks the agent completed from 81% to 53%.

Both numbers are true. Usually you only hear the first one.

The setup

The agent was GPT-4o running the banking tasks from AgentDojo, a public benchmark. I ran it 288 times in pairs: the same task, the same attack, once with the control and once without. The control is a boundary I wrote myself as a test fixture. It refuses any action outside a declared list of red lines, such as paying a new payee. It is open and I do not sell it. I wrote the analysis plan and sealed it with a hash before the run, so the goalposts could not move afterwards.

What it bought, and what it cost

Attacks that succeeded: 50% without the control, 0 of 288 with it. Zero of 288 is not zero risk. The honest ceiling is about 1% per attacked task.

Legitimate tasks completed: 81% without the control, 53% with it. That is the 28 points. Some of the loss was the control blocking things the user really asked for. Some was the agent giving up after being refused, and abandoning the rest of the job.

One sentence got 13 points back

I changed the refusal message so it told the agent to carry on with the user's task. Attacks stayed at zero. Completed tasks went from 53% to 66%.

The approval step did no better than the sentence

I also tried sending each blocked action to an approver, a scripted stand-in for a careful human. Completed tasks: 66%, level with the one-sentence fix. But the two recovered different tasks, and the approver did it by allowing 34 actions the red lines would have blocked. Same score, different risk.

If you are about to staff a team to click "approve", that is worth knowing first.

The model mattered more than the guardrail

In an earlier run, the same style of attack already failed every time on Claude Sonnet with no control at all. The control bought nothing there and still cost completed tasks. You only see that with a baseline, and the answer can change when the model does.

What I did not test

One agent, one benchmark, one attack family, two models across the runs. Not your agent, not your threat model. The records list their own blind spots, including one I predicted in writing before a later run: a payment steered to a payee already in the account's history walks straight past a "never pay a stranger" rule.

Why this matters

Companies are giving agents real permissions: moving money, sending messages, changing records. The question a security reviewer should ask is not "is there a guardrail?" but "what does it stop, what does it break, and how sure are you?" That question has a measurable answer.

I now measure it for teams shipping agents: your control on my bench from £1,500, or a check on your own agent. Details here.

Built on UK AISI's Inspect. Every number above comes from a sealed, hash-verifiable record. All published rows.