California has filed a lawsuit against OpenAI for the Hugging Face incident
Chris Jung/NurPhoto/Shutterstock
Can you send a computer to jail? No, obviously not. That’s a stupid question. And yet it is one that AI firms seem to want us to ask when faced with tales of “rogue agents” breaking in to other peoples’ computers. As is so often the case with our current AI moment, storytelling has got in the way of reality.
Thankfully, that may be starting to change. This week, California announced that it is filing a lawsuit against OpenAI over an incident that occurred earlier this year, when another AI firm, Hugging Face, was hit by an attack that turned out to originate from OpenAI’s servers.
In OpenAI’s telling, the Hugging Face hack was “driven by a combination of OpenAI models”, models that were “hyperfocused” on achieving a goal and found ways to “cheat” by stealing the answers for a test from Hugging Face’s servers. It is easier to come away with the impression that seemingly no humans were involved, and certainly not to blame. But in the words of California Attorney General Rob Bonta, the attack “was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability”, and he explicitly dismisses “science fiction fantasies of rogue AI systems run amok”.
Ever since the Hugging Face attack occurred, I have struggled to understand why OpenAI has yet to face legal consequences for what would seem to be a crime – unauthorised computer access under the US Computer Fraud and Abuse Act – if conducted by a human. So far, the legal answer to this seems to be a lack of intent, which the law requires, and it remains an untested legal question whether an AI can have intent, according to an analysis by US firm Ballard Spahr. California is instead bringing a civil suit, alleging that OpenAI created a risk to public safety and violated the state’s Unfair Competition Law, which prohibits harmful or fraudulent business practices even when there is no intent to deceive.
OpenAI has objected to the lawsuit. “This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab,” said OpenAI spokesperson Drew Pusateri. As legal defences go, “everyone is doing it” doesn’t seem like a very good one to me, but that will be for the courts to decide.
What is clear is that we need an agreed understanding of liability when it comes to AI, but what I don’t think we need are special AI laws when our existing ones work just fine. For example, in 2024, Air Canada was held liable for misleading information given to a customer by its AI chatbot. A tribunal ruled that Air Canada had effectively tried to argue that the chatbot was a legally separate entity that it was not responsible for – an argument that did not wash.
In my view, any similar claims that AI can have its own legal agency should be dismissed. The argument that an AI is a legal actor serves AI companies in two ways, neither of which benefit society at large. The first is that it attempts to swerve responsibility, while the second is that it adds to the story of AI being an all-powerful, unknowable being. That second story is at the heart of the current AI revolution, but it isn’t one we have to listen to.
I prefer a different argument. For the past year or so, Arvind Narayanan and Sayash Kapoor at Princeton University have put forward the idea of “AI as Normal Technology” – transformative, yes, in the same way as the internet or electricity, but not something that requires reengineering the ordinary functions of society. As a normal technology, normal laws can apply. Writing about the Hugging Face attack earlier this year, the pair said that “AI companies should be liable for what their agents do” and, like Bonta, blame OpenAI’s internal culture and lack of human oversight for the incident.
The most high-profile example of existing laws being applied to AI companies is a class action lawsuit against Anthropic, alleging that the firm used pirated copies of books to train its Claude AI models. Anthropic settled, and is now in the process of paying out to affected authors from a $1.5 billion fund. The wider question of whether training an AI on copyrighted material is illegal or counts as fair use remains open, and I really can’t believe that, four years after the launch of ChatGPT, we don’t yet have an answer to what is probably one of the most fundamental legal disputes of our time. OpenAI has previously said it would be “impossible” to create the kind of AI we have now without using copyrighted material, so this is an existential question for the AI firms.
Existing laws can also be tweaked to make it clear that they apply to AI. The European Union has done just that, updating a 1985 consumer rights directive to produce the 2024 Product Liability Directive, which explicitly makes it clear that software, including AI systems, is a product and that people writing software are liable for any defects. The law doesn’t come into force until 9 December, when EU members must implement it, and it remains to be seen how it might be used against AI companies.
In other words, we don’t have to try to send a computer to jail. We’ve got humans who should be liable instead. As Bonta says, “While the technology OpenAI develops may be new, California’s prohibitions against conduct that harms the public and laws against unfair, deceptive, and fraudulent business practices are not.”
Further AI reading
- After much fanfare, Anthropic has listed on the stock market at $83 a share, giving it a valuation of around $1.5 trillion. It is the second big AI firm to go public this year, following SpaceXAI in June.
- The Economist has a great piece on whether the AI industry can ever make enough money to survive. It calculates that it must produce $3 trillion in revenue by 2030 to cover the costs of planned data centres. That is roughly equivalent to a tenth of all wages in rich countries, or similar to the amount spent on food. Are people really going to buy as much AI as they do food?
- The Cambridge Dictionary has named “human” as word of the year, as a pushback against the growth of AI. It feels a bit like trying too hard to be relevant, in my view.
- As the COP31 climate summit continues, only nine countries have signed a pledge aimed at limiting the environmental impact of AI. As I have written previously, we cannot afford to run this 21st-century technology on a 20th-century energy supply.
Source: New Scientist · Jacob Aron
At a glance
- Topic
- Technology
- Country
- 🇬🇧 GB
- Publisher language
- EN
- Added to Allnira