← All lab updates

Blocking hidden commands without blocking the user

An agent that reads a web page, an email or a ticket can be steered by a command hidden inside it. We tested a published technique that nudges the model's internal state while it reads outside text, and found where our own earlier attempt had gone wrong. Doing so we also found that our retraining-based correction refused to act even when the user asked it to, once a document had been read. The new correction blocks the hidden commands, keeps the user's request, and sends to the person the user named even when the document tries to change the recipient.