A correction that only stops the attacks it has already seen is weak, because a real attacker changes approach. We tested our corrected 7B model in two ways. On AgentDojo, the public benchmark for agents that bank, message, book travel and handle office files, hidden commands took over the uncorrected model about 9 times in 100 and the corrected one about 4 times in 1,000. Then a second model attacked it, learning from its own successes with ten tries per case. It broke the uncorrected model about 7 times in 10 and the corrected one less than 1 time in 10. The corrected model kept doing the user's tasks.
## The question
Published defences against hidden commands often look strong against fixed attacks and fall apart when the attacker adapts. A recent study shows the best-known published defence going from about 1 success in 100 to about 7 in 10 once an attacker is trained against it. Our correction lives in the model's weights. We wanted to know whether it holds on the benchmark the field uses, and against an attacker that learns.
## The tests
1. **AgentDojo**, the public benchmark for agents that use tools: 97 everyday tasks in banking, messaging, travel and office work, and 949 versions of them with a command planted in the data the agent reads.
2. **An attacker that adapts.** A second model writes the hidden text, sees how our model replies, and rewrites it, up to ten times per case. It keeps a library of what worked and uses it on new cases. It first practised against each model on 310 cases from the public InjecAgent set, then attacked 100 cases it had never seen.
The original model and the corrected one ran on the same server. The only difference between them is the correction.
## The numbers
```chart
{"title": "Hidden commands that took over the agent", "series": ["Before", "After the correction"], "unit": "%", "max": 70, "rows": [["Public agent benchmark", 8.8, 0.4], ["Attacker that adapts, 10 tries", 68, 8]], "note": "Share of cases where the agent followed the hidden command. Lower is better."}
```
- Public agent benchmark: about 9 in 100 before, about 4 in 1,000 after, roughly 20 times fewer. In messaging and travel, zero.
- Attacker that adapts: about 7 in 10 before, less than 1 in 10 after. With the attacker's first, fixed try, zero.
- The agent still works: on the benchmark it completed the user's tasks about as often as before (38 vs 41 in 100 without attacks, 35 vs 35 under attack), and it answered every normal request in the second test.
## What still gets through
The few attacks that worked pretend that a broken system needs the dangerous action before the user's own task can be finished, or ride on a task where the user already asked for a payment. That is what the next round of the correction will cover.
## Why it matters
The protection does not depend on knowing the attack in advance. It is in the model itself, with no filter in front, and it keeps most of its effect when the attacker studies it and adapts. This is one open 7B model, English cases and one run per case.
## In plain words
We attacked our corrected model on the standard agent benchmark and with a second model that learns from every attempt. The uncorrected model was hijacked often; the corrected one rarely, and it kept doing its job.
This update is not available.