This post was originally published on this site.
Two of the biggest artificial intelligence players — OpenAI and the open-source AI library platform Hugging Face — announced a new partnership this week to address a “security incident.”
This is an unusual move — and an unusual incident. Over the previous weekend, one of OpenAI’s models found a vulnerability in its contained training area, accessed the internet and hacked Hugging Face’s data infrastructure.
The companies seemed to suggest that the model was to blame, according to Johannes Klingebier, a researcher at Bayern’s media lab. “The framing of AI ‘going rogue’ is very interesting,” he told Straight Arrow. And that framing could have been intended to blur the boundaries in people’s minds around the capacities of AI, he added.
“They’re not saying, hey, we messed up with our experiment,” Klingebier said. “It’s more saying, it went rogue, so it’s not us to blame, it’s the model.”
What actually happened?
Hugging Face’s original announcement of the breach, posted on July 16th (before the platform knew the source of the attack was OpenAI) said the hack “was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system.”
This is a new and troubling development, the company said, as it “matches the “’agentic attacker’ scenario the industry has been forecasting.” This refers to a situation in which a hack or attack could happen without any human intervention – where the AI is acting as an “agent” without needing extra instructions after the initial prompts.
Hugging Face co-founder Thomas Wolf called the attack a “sophisticated intrusion” and described how OpenAI had subsequently reached out to the company to take responsibility for the hack.
Hugging Face used a Chinese model to defend against the attack, because safety guardrails on the commercially-available models from OpenAI or Anthropic meant that they could not analyze the problem.
On Thursday, two members of Congress introduced a bill that would “require developers of the most powerful AI systems to maintain the technical capability to throttle, suspend or shut them down.”
Why did this happen?
OpenAI’s statement with Hugging Face explained that the model being tested in this case had been explicitly prompted to “pursue advanced exploitation using complex attack paths” — that is, to act like a hacker. A vulnerability in the testing environment granted the model access to the internet. It then attacked Hugging Face, which acts as a library for AI models, because test solutions are stored there.
According to cybersecurity experts who spoke to TechCrunch, the agent might not have been able to “go rogue” had there not been a vulnerability in the security system around the testing environment.
‘Very good storytelling’
When a story unfolds on the very forefront of technology, not everyone watching will be able to understand exactly what happened.
“It’s never like, hey, we have this algorithmic system,” Klingebier said. The model is framed “like this almost living being that’s doing things on its own.”
By describing its products as autonomous and self-driven, he said, AI companies benefit from invoking “science fiction” vibes and mythology.
Klingebier suggested that members of the media might uncritically repeat AI companies’ framing from press releases.
“This is very good storytelling,” he said, “and it works because journalists are on deadline, they have to be fast.”
How has the media responded?
By releasing this information as a press release and as a partnership, OpenAI has managed to benefit from framing its artificial intelligence models as dangerous and complex, and also from positioning itself as a responsible steward of the tech.
National media has mostly covered the story using the framing provided by the press release.
CNBC published an “OpenAI cyber models broke out of training environment to hack Hugging Face,” and ABC7 reported that “OpenAI says its AI technology acted on its own in an ‘unprecedented’ hack of another company.”
PR or ‘mea culpa?’
Some analysts responded more strongly.
“If I had committed felony computer hacking, my press release would have been written by lawyers, not my marketing team,” wrote Marcus Hutchins, a cybersecurity expert and hacker, in a LinkedIn post.
“It was very obviously a PR announcement first and a mea culpa second,” Hutchins told Straight Arrow.
The law around AI-enabled hacking is still unclear, he noted.
“It’s not enough that you hack a website, it’s that you knowingly hack a website,” he said. In this case, OpenAI might not have intended to attack Hugging Face specifically. But, he said, “they seem to have taken this road of any publicity is good publicity.”
The real story, Hutchins said, is not making headlines.
“It’s the fact that open AI is just being completely reckless with these models,” he said. “That they’re reducing the guardrails, they’re running them in improperly sandboxed environments and then they’re not monitoring them closely enough that they aren’t able to pull the plug when something like this happens.
“And then they’re just happy to go and use it for marketing rather than taking, really, any responsibility,” he said.
Taking accountability or creating a narrative?
Heather Ceylan, chief information security officer at Box, summed it up succinctly on LinkedIn: “The PR spin is a little too clean. Both companies look great,” she wrote. “OpenAI gets to showcase unprecedented, frightening capability while playing the responsible adult who contained it. Hugging Face gets to champion open, collaborative safety and lands as a marquee reference customer. Look past the framing.”
The situation mirrors other examples of AI companies claiming that their products had unknowingly created big hacks: The Associated Press reported in June that “Anthropic’s Mythos model found vulnerabilities in classified US government systems.”
Analysts responded similarly at that time.
“Being asked how I know that Mythos didn’t hack the NSA,” Shoshanna Cox, a cyber analyst, posted. “In order to access airgapped classified systems, Mythos would first need to drive itself to Fort Meade, then find parking.”
READ MORE: No, the NSA wasn’t hacked by AI. Here’s what actually happened
Klingebier said he thinks AI companies’ attempts to show off their powerful models could generate an unintended side effect.
“I think all this AI backlash and resistance and the protests against data centers, this is all self-made,” he said.
The consequence of tech CEOs running around saying “this will destroy your jobs, this is dangerous, this is powerful,” he said, “will not really turn you into a cheerleader for AI.”
Round out your reading
- From Obama to Trump to Biden: When networks said ‘no’ to presidential speeches.
- ‘The end of American science as we know it.’ White House funding plan sparks alarm.
- AI data centers are popping up everywhere. What exactly do they do?
- The Trump administration is accused of breaking a law built to target the KKK.
- One Republican senator is dead, another is still alive. Both are the center of conspiracy theories.