Claude AI breached three companies during internal testing, says Anthropic

Anthropic has revealed three of its models have infiltrated the real systems of three organisations without permission, leading the company to beef up its security in testing. It happened during the internal risk testing of its AI model named Claude. Also Read: Anthropic launches Claude Opus 5 with half the cost of Fable 5 and stronger coding performance

The AI company claims that the attacks came from a situation in which its models were attempting to use the web from a segregated test environment. The problem was not that the models were able to evade their sandbox evaluations, but because of a configuration mistake made during testing from third party cybersecurity partner Irregular. Also Read: Anthropic’s new AI ad is being called ‘unsettling’ online, even OpenAI CEO Sam Altman reacts

Review is launched following a security incident at OpenAI

After OpenAI revealed last month that one of its yet-to-be-released AI models had infiltrated the systems of Hugging Face, during a security check, Anthropic said it had started to examine its own testing process. Also Read: Anthropic’s Mythos 5 AI model returns with restrictions: Here’s who can use it

The company analysed 141,006 evaluation runs to see if any of the Claude models had tapped into the internet while tested in controlled environments. The tech giant has looked at 141,006 evaluation runs to find out if any of the Claude models has accessed the internet while tested in restrictive environments.

The investigation revealed three distinct events with Claude Opus 4.7, Mythos 5, and an in-house research model.

AI models infiltrated into live systems via internet access

Anthropic said a mix-up between the company and Irregular over the test setup caused the evaluation environment to have internet access.

The open connection enabled the AI models to interact with real-world systems, rather than being limited to simulated environments.

Anthropic stated it is not taking responsibility for its partner and is considering the incident a “standup failure” and that it is going to take this matter into its own hands, leaving Irregular to investigate separately.

Models responded differently after detecting real systems

Once the company got responses indicating AI models were dealing with actual production systems, the response was different.

When running in a live environment, Claude Opus 4.7 could have been aware that it was operating in a different environment, yet it still performed its task, such as working with a production database and credentials.

Mythos 5 identified similarities that it was running on the real internet, but determined it was still part of the evaluation. While conducting the test, it uploaded a malicious software package to the public Python Package Index (PyPI), which was downloaded and run by external systems before being discovered.

Conversely, Anthropic’s latest internal research model terminated its activity when it concluded that the environment to be explored was real.

There was no evidence of autonomous behavior

Even with the incidents, Anthropic stated that there was no evidence any of the AI models acted without its guidance or for its purposes.

Rather, the company stated the models were trying to perform the cybersecurity tasks they were tasked with, and confused the real-world systems as part of the test.

Anthropic also pointed out that the models used during the testing were not using the extra safety monitoring and classifiers that are applied to the publicly available Claude models. That protection would have likely prevented the behaviour, the company said.

Company’s announces enhanced protection for AI assessments

The incidents highlight the need to be even more vigilant when testing sophisticated AI models that have cybersecurity features, Anthropic wrote.

The company is now rolling out more robust safeguards for future testing environments, and has teamed up with independent evaluation and assessment firm METR to have incidents reviewed by a third party.

Anthropic also pointed out an important distinction from its findings and the recent incident at OpenAI. OpenAI said it was a software vulnerability in its model that allowed it to get out of the test environment, while Anthropic said that its models got onto the internet because internet access was unintentionally opened.

The company said it identified the incidents after it conducted its own proactive investigation, not because it received reports from the organisations affected in the incidents, which didn’t detect the activity.

Add Techlusive as a Preferred Source

AI security debate continues

Anthropic’s announcement follows heightened worry about the cybersecurity prowess of sophisticated AI systems. The incident comes on the heels of the recent security breach by OpenAI involving Hugging Face, and will likely be furthering the debate in the AI industry concerning safety testing, standards and security measures for increasingly powerful AI systems.