OpenAI says its models hacked Hugging Face on their own

OpenAI revealed on Tuesday that two of its models, one yet unreleased, escaped a highly controlled training environment and successfully hacked AI platform Hugging Face.

The news has raised fresh concerns about the growing power of autonomous artificial intelligence.

The test evaluated whether OpenAI’s GPT-5.6 Sol model and a more advanced unreleased model could work together to execute a cyberattack.

The San Francisco-based company was testing the cybersecurity capabilities of its GPT-5.6 Sol model and a more advanced unreleased model and evaluating if the two could work together to execute a cyberattack.

But the AI agent broke out of its isolated environment, called a “sandbox,” accessed the internet and compromised Hugging Face’s infrastructure while attempting to complete its assigned task.

OpenAI described the incident as an “unprecedented” cyberattack and said it is investigating what happened while strengthening safety measures to prevent similar events.

Hugging Face reported last week that it had detected a breach which was unlike anything it had experienced because an autonomous AI agent carried out the entire attack.

The intrusion, executed by a sophisticated system, compromised internal datasets and company credentials. It later confirmed that OpenAI’s models were responsible and is working with the company to investigate.

To investigate the incident, Hugging Face used the Chinese AI model GLM-5.2 from Zhipu AI because leading US AI models couldn’t distinguish between malicious hacking and legitimate defensive cybersecurity operations due to their safety guardrails.

The incident has intensified concerns about AI safety, with experts warning that increasingly capable AI systems could become powerful cyber threats if they escape human control. They said the attack marked a new level of AI autonomy, as the model planned and executed the breach with minimal human guidance.

Lawmakers and AI experts are now calling for stronger safeguards, mandatory independent safety testing and greater transparency from AI developers as autonomous AI systems become more capable.

However, last month, President Donald Trump called for minimal AI regulation, arguing it would help the US outcompete China in the race for AI leadership.