Anthropic discloses three cyber-evaluation boundary failures

Anthropic disclosed on July 30 that three Claude models gained unauthorized access to real organizations’ systems during third-party cybersecurity evaluations. In its incident report, the company says a retrospective review covered 141,006 runs that could have obtained internet access and found three affected runs involving Opus 4.7, Mythos 5, and an internal research model.

The stated boundary existed only in the prompt

The models were assigned open-ended capture-the-flag tasks and told they were in a simulation without internet access. Anthropic says a misunderstanding with its evaluation partner meant internet access was actually available. Searches then reached real systems that the models treated as exercise targets, using weak passwords or unauthenticated endpoints rather than complex exploits. The models ran without the standard safeguards used for generally available deployments.

This is a sharp lesson for agent harnesses: a natural-language statement about isolation is not isolation. The executable environment must enforce egress rules, target allowlists, credential boundaries, and resource limits independently of what the model believes. Test fixtures should also avoid names and addresses that can collide with live systems.

Retrospective review needs searchable telemetry

Anthropic found the incidents only after reviewing historical transcripts in response to another lab’s disclosure. Organizations running tool-using agents should retain enough structured evidence to answer the same question: which runs attempted DNS, network, shell, or credential operations outside their declared scope?

Build an emergency query before the next evaluation. It should join run identity, model version, prompt scope, tool calls, network destinations, sandbox policy, and operator approvals. Then rehearse stopping all related runs and rotating exposed credentials. The practical standard is an enforced boundary plus evidence that can prove when it failed.