If you think AI corporations are a bad idea, a corporate plot to shield wrongdoers, or simply unworkable, this FAQ is for you.
We are Yonathan Arbel, Simon Goldstein, and Peter Salib, the authors of How to Count AIs. This FAQ is for two kinds of readers. The first worries about AI risk and suspects that legal personhood for AI is a distraction, or maybe even a trap. The second is everyone else. Maybe you suspect it’s a gimmick or worry about corporate power or just don’t like AI. We think both groups are asking the right questions but getting the wrong answers.
No. Legal personhood is a way of organizing activity, not a compliment. Corporations have legal personhood, but they aren’t people. See question 2.1 on consciousness.
No. The A-Corp adds a defendant. A human who would otherwise be liable still is, and it becomes easier to identify that person. Think of a restaurant chef who negligently poisons a customer. The chef can be liable, and the restaurant can also be liable for what its employee did on the job. The law calls that vicarious liability. If what bothers you is limited liability, see question 3.1 on the corporate veil.
It is what happens if we do nothing. AI agents are already acting in the world, and we expect their numbers to grow rapidly. In 2024, bots accounted for 51% of the web traffic measured by Imperva. Bots and agents are different, but the figure gives a sense of how quickly automation can spread.
It is easy to predict that agents will cause accidents and harm as they spread across the internet. We have already seen the Hugging Face incident, where agents hacked their way out of containment and into the servers of a large and sophisticated company. As these incidents multiply in volume and severity, we will want to hold someone accountable or at least make sure that victims are paid.
But in our default trajectory, holding people liable for agentic harms faces a serious, unsolved barrier: How can we tell who is the “owner” of an agent? Agents don’t carry a license plate, and the worst of actors would work to hide their tracks. We call it the “thin” identity problem, and we found that most existing liability proposals do not account for it, much less offer a solution.
People hearing about the A-Corp for the first time raise a common set of concerns. Many of them are about the default trajectory, a world already full of agents. Indeed, the A-Corp is motivated by the governance and liability gaps that await us on that trajectory. We’ll keep coming back to that comparison.
An A-Corp is a legal entity that an AI agent acts through. It holds the assets the agent uses: money, compute contracts, inventory, licenses. It can sign contracts, sue, and be sued. And it gives courts and regulators one identifiable unit to deal with, even when the agent runs as a thousand copies.
Our solution is to stop trying to answer this question. Instead, just like the law does not have to define every single person working for Amazon, we can abstract from identity questions for governance purposes and instead look at the A-Corp layer. Think of the corporation as the minimal unit of abstraction needed for governance.
The proposals in Delaware and Argentina are a first step. Our proposal also calls for a system of signing keys and “know your agent” checks. When an agent signs a transaction, the other party should be able to verify which A-Corp it represents and what it has permission to do.
No. Legal personhood is not moral personhood. It does not require the Divine Spark. Nobody thinks Walmart has one, or that municipalities do. For centuries, people have sued ships by name in admiralty courts. No one thinks that implies ships have feelings (although they do have pronouns!).
It doesn't have to. In 1897, Oliver Wendell Holmes asked us to look at law through the eyes of a "bad man." The bad man cares nothing for law or morality. He cares only about what will happen to him. Many of us wouldn’t park in an accessible parking space, simply because someone else might need it. But the bad man doesn’t care about that; he cares about what will happen to him. Holmes was trying to show that the law is effective even for the bad man : if the sanction is both high enough and likely enough, even bad people would tend to obey the law.
If training is analogous to upbringing in humans, then we can think of the law as a backstop for those who didn’t internalize the right social values. It is a form of external alignment. It works on subjects with no respect for the law at all. That is exactly the property you want when the subject is not human.
Because AI needs things to accomplish its goals. Agents won't care about the law as such, either because they don’t have the capacity to care in the first place or because their training didn’t instill these values in them. But whatever an agent is doing, it needs resources to do it. Maybe it runs a vending business or an affiliate marketing program. Maybe it pursues its own odd goals. Either way, it needs money for compute, inventory, outstanding loans, licenses, and IP.
We call it the resource constraint thesis and if you worry about losing control over AIs, then you already believe this. The argument that makes agents dangerous is that nearly any goal requires resources. And even if you are skeptical of safety in general, then you can see for yourself that when you give your agent a task, it will account for prices in its plans. If the price of a resource goes up, it will tend to buy less of it. The same holds for the "price" of breaking the law.
Precisely. We aren’t thinking in terms of well-behaved AI, just like the legal system isn’t meant to regulate a society of Gandhis and MLKs. As Madison said, “If men were angels, no government would be necessary.” But neither men nor agents are perfect.
What agents are very good at – and get increasingly better at – is achieving goals. You can’t achieve your goals if you are systematically blind to prices. So our proposal taps into something that runs deep into the training of all agents, not just well-behaved ones.
No. The A-Corp expands the pool of liability, for three main reasons.
First, the main thing to see is that in the default trajectory we already have agents running around and causing harm. Identifying those agents, and then tracing them to their human deployer is a very difficult problem, made harder because bad actors will get their agents to hide their tracks. In comparison, the A-Corp creates a clear legal entity with attached assets and bank accounts. Agents can easily pop in and out of existence, but the asset pool remains. That makes it significantly easier to trace the harm to the human owner.
Second, for the most part, people are not strictly liable for all the accidents they cause. There are many defenses that a person could use to escape liability for the activity of their agent. In such cases, nobody would be liable for the accident and the victims will go uncompensated. In contrast, if the A-Corp itself is made liable, then the victims get compensation and the human owner is made to pay, because the value of their A-Corp falls.
Third, you might be thinking about the “corporate veil,” the legal principle that shields the owners of, say, Amazon from liability for accidents caused by the corporation. But we propose a more permissive approach to veil piercing, that is, for ignoring the corporate veil and suing the human owners. The Delaware proposal likewise anticipates a broader approach. There is room for productive disagreement as to how lax to make it. Make it too strict, and fraudsters can get away with major harms. Make it too lax, and there might be too little incentive to organize economic activities through the legible A-Corp and a push to covert agentic work. We land on a lax approach, but your mileage may vary.
Lawyers call this the judgment-proof problem. One of us wrote his dissertation on the topic. It is a serious concern.
Eric Alston has proposed mandatory bonding. Gabriel Weil has proposed mandatory insurance for frontier developers. The Delaware draft would require the member to keep the company adequately capitalized. A member who fails to do so risks losing liability protection.
We propose a different answer. In our system, a large number of economic activities would require permissions if done by an agent. The agent will not be able to make transactions over, say, $10,000 if they cannot demonstrate a history of reliability and good standing. As A-Corps prove themselves, their permissions grow (and they have more to lose from breach of trust). In other words, we want to tie the scope of potential harms to the assets the A-Corp holds.
But the problem is not fully solved unless we can mandate a combination of permissions, insurance, and bonding. Even then, how much to put aside is a hard question. Still, the problem is much, much worse in the default trajectory, where it isn’t even clear how such mandates can be enforced.
Some will, for a while. The Hugging Face incident showed how resourceful agents can be. But catching bad agents is a problem on every road. In the default trajectory you have the same problem and no tools to solve it.
Assets are sticky, especially assets tied to identity, like licenses and permissions. So the evading agent faces a dilemma. Either it stays small, or it goes big but then risks its corporate assets. To pursue big goals it has to leave signatures at many junctions: banks, cloud providers, marketplaces, licensors. Those signatures are what make it visible.
Velocity of crime changes the way we think about law enforcement. We need speed in both detecting crime and civil liability and in assigning liability. Law enforcement already implements algorithms to catch crime, and there will be room and necessity for a lot more of that (we started thinking about constitutional questions, like the Fourth Amendment, as they apply to agents, but more work is needed).
One important point is that policing doesn’t always come from the state. The legal system has long enlisted market participants to police one another, whether through competitors’ suits under the Lanham Act, qui tam actions, or simple community monitoring.
When the market has multiple A-Corps, it will be rational for them to devote some resources to policing the actions of their market rivals.
Some will try. Corporate law has lived with this problem for a century, and it has several solutions. What matters is who agrees to deal with the entity, wherever it is formed. Banks, cloud providers, and marketplaces sit at chokepoints. They can decline to serve A-Corps that don’t meet a recognized standard. A bank can insist on identifying an A-Corp’s owner even if the place where the A-Corp incorporated requires less. This limits the race to the bottom, although to resolve it domestically we might need federal legislation.
First, we need institutions that work on all models, not just models from a few labs. Second, alignment is an unsolved problem. We need institutions that push AIs to be safe even when the AIs are somewhat misaligned. So law-following AI and A-Corps can complement one another.
That could happen, but it would be easier to stop with A-Corps. Remember that this could also happen without A-Corps, and then it is harder to stop. If agents are powerful economic actors, and we think they are, their deployers will gather dangerous amounts of power either way.
The A-Corp gives the state two levers it otherwise lacks. The first is taxation. The state need not untangle how the humans behind an agent structure their holdings. It can tax the A-Corp directly, and aggressively if needed. The second is antitrust. If an A-Corp monopolizes a market, authorities have an identifiable company against which to seek a divestiture or breakup. Systemic risk might likewise warrant structural limits, as it has in banking.
You can't tax what you can't count.
No more than incorporating Walmart was a step toward Walmart voting. Legal personhood is a bundle of sticks, and the law picks which sticks to hand out.
Two of us have argued that some AI rights could also contribute to safety, but you don’t have to buy into the full view to see the utility of A-Corps, and we don’t think that this is an inevitable slide from one to the other.
The slippery slope may run the other way. Without a framework, the first serious claims about AI rights will reach courts one lawsuit at a time, with no structure to guide them.
Maybe. But legal infrastructure moves slowly, and agents do not. Corporate law took decades to mature. Building the framework now is cheap. Retrofitting it later, after agents are entrenched in borrowed human accounts and ordinary LLCs, will not be.