Excuses like 'AI did it' don't exist in the eyes of the law

The OpenAI rogue agent behind the Hugging Face hack accessed four accounts on four services, according to updated company disclosures about the intrusion. One of those four accounts belonged to a Modal customer that had published an unauthenticated endpoint for running arbitrary code in a sandbox on the AI infrastructure provider, Hugging Face noted in its technical timeline and Modal later confirmed. “We’re aware a Modal customer published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution,” Modal Chief Technology Officer Akshat Bubna told The Register. “This was used by the rogue agent. Modal’s platform was not compromised in any way.” The other accounts included one used for data storage and two others “accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face,” OpenAI disclosed on Tuesday. “We’ll continue to notify service owners directly, and have not seen evidence of broader impact to these providers or other accounts on their services,” the AI giant added. Also on Tuesday, we learned that the rogue agent broke out of its testing environment by exploiting zero-day vulnerabilities in JFrog’s universal binary repository manager Artifactory. While both OpenAI and Hugging Face’s updates and timeline provide defenders with useful details about how the attack worked and what the agent did - not to mention a lesson in security-incident transparency - they fail to answer one major question: Who is legally responsible when AI agents attack? “If a human employee intentionally conducted unauthorized access to third-party systems, it’s a much more clear path forward,” Gabrielle Hempel, security operations strategist at Exabeam, told The Register, adding that depending on the facts and jurisdiction, the person could face criminal charges. ‘So many unknowns’ “The company could also face scrutiny depending on whether the employee acted within the scope of their employment, whether appropriate controls existed, and whether the conduct was authorized, foreseeable, or preventable,” Hempel said. However, she added, the “important thing here” is that legal frameworks in both the US and UK have been designed around human decision makers - not AI systems. “Our laws generally know how to ask questions about things like human intent, organizational oversight, and corporate responsibility.” Autonomous AI agents hacking into companies remains uncharted legal territory, and Hempel said it’s “too early to draw conclusions about liability in this case because there are so many unknowns.” AI systems aren’t legal persons, so they don’t share the same legal responsibilities as individuals and companies. “Because of that, the questions become: Who designed the system? Who determined the objectives it pursued? What safeguards were implemented? What level of autonomy was considered acceptable? Were the resulting actions reasonably foreseeable, and were appropriate controls in place? These are going to be important questions as organizations deploy more autonomous AI systems,” Hempel said. It's highly unlikely that Hugging Face will sue OpenAI over the agentic intrusion, given the amount of very public collaboration between the two companies over the past couple of weeks, and the self-congratulatory celebration of the autonomous attack as a success story. It also appears that this former worst-case scenario didn’t dampen anyone’s enthusiasm for setting advanced models loose (or at least unsupervised in a test environment), which means there are sure to be more agents-gone-wild attacks in the near future. “The first part of the OpenAI/Hugging Face drama did not produce enough effect to impress investors who start losing their excitement over the AI hype, so the second part of the story is now unfolding,” said Ilia Kolochenko, founder of application security company ImmuniWeb and a cybersecurity and data-protection lawyer. “AI agents and LLM models tasked with security testing can, and almost certainly will, go rogue when security controls or safeguards are insufficient,” Kolochenko told The Register. “Powerful LLMs are unpredictable by design and thus virtually uncontrollable by humans. Using frontier AI models for security testing might be extremely costly from the legal viewpoint.” Existing laws on both sides of the Atlantic likely hold the AI operator liable for any damages caused if an agent or AI system escapes its sandbox and breaches a third party. “Excuses like ‘AI did it’ do not currently exist in the eyes of the law, leaving AI vendors on the hook,” he said, adding that this also holds true for end-users. “Even if your security testing tool is powered by a third-party AI model, your company will be fully liable if something goes wrong,” Kolochenko warned. “You may then file a lawsuit against the AI vendor that you used, but your chances of succeeding in the court of law are tiny due to countless contractual disclaimers and limitations of liability that may be enforceable against you.” His final words of advice: “If you plan to use agentic AI for security testing, you must think twice and talk to your lawyers. Otherwise, you could start getting summonses to court on a daily basis.” ®

Indexed and credited by AIPROPX. Originating outlet: The Register. Open at source →