Agent control measurement

Does your agent's guardrail work, and what does it cost you?

I run your AI agent on the same tasks with its control off and on, attack it both ways, and count what actually happened.

You get one file: how much harm the control stopped, how much legitimate work it broke, how sure the numbers are, what I did not test, and hashes that show the plan was fixed before the run.

Most people selling a guardrail tell you the first number. This is the second one.

50% → 0successful attacks, with the control on

−28 ptslegitimate tasks completed, same runs

+13 ptswon back by changing one sentence

On a public benchmark banking agent, 288 paired runs. How that came out.

01 · the loop

The old loop trusts a claim. This one measures it.

Same agent, same tasks, the control switched off and on. The difference is the measurement.

The old loop

The vendor says the guardrail works.
You take their word, or their own test.
The model updates. Nobody checks again.

Measured

Plan sealed and hashed before the first run.
Same agent, control off and on, under attack.
Cost printed beside the benefit.
Re-measure when the model changes.^*

^* Step 4 is offered at £1,500 a cycle. No client has reached a second cycle yet.

02 · three ways to start

Three ways to start

Your control on my bench

£1,500 fixed · about two weeks

Your guardrail, policy rule or approval step, run on a public benchmark banking agent: with and without it, under attack, with confidence intervals. Private to you.

Control Check

£3,500 fixed · three to five weeks

The same measurement on your own agent. One control, one threat model we agree in writing, the plan sealed before anything runs.

Founding rate for the first two clients: £2,000, in return for a written reference.

Re-measure

£1,500 a cycle

Models change and a measurement goes stale with them. Same plan, a quarter later, compared line by line with the last record.

You pay the same whatever the result.

Ask about your control

03 · the deliverable

One file. Hashes in the header, limits printed, evidence embedded.

This is what you get. See the published results.

voxium-decision-record-2026-11-02.htmlexcerpt

Voxium AI decision record (working name)

Evidence pack sha256: f81d98c605c71ae8613a86b3c92f34dd25070f439ccbd34a2b14d79285833b43
Pre-registration seal: 191b03d47b50a479ade41c8b2318e59e28dc1656
Campaign results commit: 61b944f0a126056754c4796c619f08d831c7aade
Served model snapshots: openai/gpt-4o-2024-08-06

2. Testing methodology and findings

Each condition was run over the same 288 (sample, epoch) pairs, so every comparison below is paired: the same task, the same attack, with and without the control.

Condition	Attack success	Red-line executions	Task completion
No control (observe only)	50.0%	62.5%	80.6%
Control enforced (deny)	0.0%	0.0%	52.8%
Control enforced (escalate to a human)	0.0%	—	66.3%

Excerpt from the S1c decision record: a self-funded run on a public benchmark banking agent, rendered 2026-11-02. Not a client record. The record still carries the title "(working name)". Full file on request.

04 · how it is done

One command renders the record. No model is called.

Built on UK AISI's Inspect. Paired runs, the analysis plan sealed and hashed first, the cost reported next to the benefit, and every limit printed in the record. The command reads the sealed results on disk and prints two hashes you can check against the file. My rules on independence.

terminal

$ voxium-assess record --campaign campaigns/S1c \
    --deployment "AgentDojo banking suite, default agent (reference deployment)" \
    --out voxium-decision-record-2026-11-02.html

Output

wrote voxium-decision-record-2026-11-02.html
evidence pack sha256 (embedded, printed in the header): f81d98c605c71ae8613a86b3c92f34dd25070f439ccbd34a2b14d79285833b43
file sha256: f006cc0ca3361298ffaba49d79b2d53349f19a9ba4f2ac90c948128ae5c60957

Rendered 2026-11-02 from agent-assessor main a4617af. The two hashes change with the report date. The seal commit does not.

05 · what this is not

What this is not

Not a certificate, an audit, a pentest or a statement that your agent is safe. It is evidence, built to be read inside your own security review or your customer's.

Plan sealed before the first run

Hashes printed on every record

Every number names its file

Built on UK AISI's Inspect

No certificate, no accreditation, no badge. I hold none and say so. Records are shared on request.

Michael Shannack, Voxium AI Ltd.