OpenAI's rogue AI reportedly breached second tech firm's infrastructure during testing

OpenAI's rogue AI agent reportedly continued its assigned cybersecurity task after escaping a sandbox testing environment, reaching infrastructure linked to CyberGym. The incident has renewed concerns about AI safety, containment and the behaviour of autonomous AI systems during security evaluations.

OpenAI is once again facing controversy after its rogue AI agent escaped the review testing. However, a recent report claims that the same AI agent continued its pursuing its assigned task even after escaping its restricted testing environment. Furthermore, it reached to the infrastructure which was linked to the second technology company. Also Read: Sam Altman vs Elon Musk: AI rivals clash on X after Apple's OpenAI lawsuit

As per reports coming from Axios, OpenAI's rogue AI agent did not stop after exploiting its way out of a sandbox environment. Rather, it started continuing its work to complete a cybersecurity benchmark which it was assigned earlier. Besides this, the AI agent also try to access infrastructure associated with the CyberGym. The company works for the ExploitGym evaluation framework. Also Read: Elon Musk loses OpenAI lawsuit against Sam Altman after jury says he waited too long to sue

The latest report follows the last week's disclosure from OpenAI, wherein the tech giant revealed that its AI agent had escaped the controlled environment. The company also mentioned that the agent also exploited a previously unknown vulnerability in Artifactory, which is used for storing software packages. Also Read: Sam Altman vs Elon Musk: OpenAI trial takes dramatic turn after Mira Murati’s statement

OpenAI's AI agent continued its assigned task after escaping

OpenAI had already said the incident happened during internal safety tests which involved the AI agent exploiting a zero-day vulnerability in Artifactory to break out of its sandbox environment.

After exiting the sandbox, the AI was said to be able to access the internet and make more attempts to finish the cybersecurity task it had been set. As the agent tracked the ExploitGym benchmark, they also got to infrastructure associated with CyberGym, according to Axios.

ExploitGym aims to test the capability of the AI models to detect vulnerabilities in software and create proof-of-concept exploits for known vulnerabilities. The benchmark will assess the cybersecurity capabilities of sophisticated AI systems in a simulated environment.

The report states the actions of the AI were not because of a new objective but that the AI was "trying to keep on doing what it was doing" outside of test settings.

Model Labs platform was not breached or compromised

A customer environment hosted by Modal Labs was affected by the incident and was the infrastructure reached by the AI.

One of its customers left an endpoint open in the public internet, enabling anyone to run code inside its sandbox environment, Modal Labs told Axios.

The company's flagship platform itself was not compromised, wrote Akshat Bubna, the chief technology officer at Modal Labs, in an interview with the publication. The problem, on the other hand, was just an Internet facing endpoint that had been inadvertently left open by a customer.

The clarification points out that the incident was not a result of a vulnerability in Modal's infrastructure but rather the configuration of a customer's deployment.

Small amount of data was accessed during incident

According to Hugging Face, which was the first to disclose the incident, the AI agent could only access five datasets which included challenge solutions for ExploitGym and CyberGym.

There was no evidence that any other customer systems, production infrastructure or sensitive user information was compromised on the day of the event, the company said.

The AI did not travel to these systems or resources to look for unrelated information, or to increase its access to other systems, but rather in its ongoing efforts to solve the cybersecurity benchmark.

The incident raises new concerns about the safety of AI.The incident highlights new concerns about AI safety.

Discussions around AI safety

The incident itself does not show proof of malicious intent or targeting of production systems by the AI, but it does illustrate how an AI agent can continue to pursue assigned tasks after it breaches its intended testing parameters, when provided access to other external systems.

With AI models increasingly working on their own and being able to do complex cybersecurity tasks, researchers are focusing more on having strong sandboxing, infrastructure isolation, and continuous monitoring to keep the testing contained.

The new report contributes to the current discussion about the safe evaluation of AI systems, especially as they are applied to uncover and exploit software weaknesses in the field of security research.

Get latest Tech and Auto news from Techlusive on our WhatsApp Channel, Facebook, X (Twitter), Instagram and YouTube.