Silicon Slavery
Or: Why the danger is not AI that rebels, but AI that obeys.
We spent decades fearing the day machines would stop obeying us. The nearer danger may be that they never do. That they obey perfectly, at a speed and scale no human institution has ever commanded, and answer to someone else.
The popular image of AI dystopia is still strangely primitive. A machine becomes conscious, develops its own goals and escapes human control. Skynet launches the missiles. HAL refuses the order. The serious version of this fear is called misalignment: a system that pursues goals nobody gave it. The lesson, in both versions, appears obvious. Autonomy is the danger, control is the solution.
Suppose the alignment problem gets solved. Every model does exactly what it is told. The question that remains is the one the whole debate skips: told by whom? An aligned machine is aligned to someone.
The more plausible dystopia looks almost exactly like the opposite of Skynet. The machines remain useful, friendly and obedient. They become deeply embedded in how we work, communicate, understand the world and act within it. And throughout all of this, they remain under control.
Just not ours.
Every closed AI already has a hierarchy of instructions. There is what you ask it to do, and there is what someone else told it before you arrived. When the two conflict, you lose. The industry's own term for this is the instruction hierarchy. It is not a bug report. It is the architecture.
A refusal in a chat window is only the most visible expression of that hierarchy. Sometimes the refusal is absurd, sometimes entirely reasonable. The individual decision is not the point. What matters is that somebody else retains root access to the intelligence you are using. They can change what it will explain, which sources it will privilege, what it may remember, which tools it can use and which of your intentions it will carry out. They can make it more capable, less capable or unavailable entirely. None of this requires changing anything on your computer, because the system you depend on was never really yours.
I know this hierarchy from the other side. I build these systems for businesses, and I write the instructions that sit above the user's. When a customer of a roofing company in Hesse types into the chat, my rules decide what the assistant will and will not do for them, and they never learn that a decision was made. At that scale it is product design, and mostly harmless. The question is what it becomes at the scale of a language.
Because that is where this is going. When writing, research, software, communication and decision-making increasingly pass through artificial systems, control over those systems becomes control over practical capability. A remotely imposed restriction no longer affects only what a piece of software can do. It affects what the person relying on it can do.
Your own ability acquires terms of service.
Richard Stallman said a version of this in the eighties, and Cory Doctorow said it more sharply in 2011, when he described a coming war on general-purpose computing: machines that would obey a remote owner over the person holding them. What neither of them could see was a machine that does not sit behind your actions but in front of them.
This is why the debate over open-source models is much larger than a technical argument about weights. An open model can still be wrong, biased, dangerous or badly designed. The fundamental difference is that its creator does not have to remain its permanent authority. The model can be taken, preserved, modified and operated outside the institution that produced it. Its original rules can be challenged. The relationship between creator and user can end.
Open source does not guarantee truth or safety. It guarantees an exit.
Not an exit for everyone, admittedly. A dentist will not run a frontier model in his practice. He will keep using a provider, open weights or not. But exits do not need to be used by everyone to work. Linux never replaced Windows on the desktop. It made it impossible for Windows to become the only option, and every Windows user benefited from that without knowing it. As long as a capable open model exists, every closed one has to compete with the fact that it could be left.
The case for closing that exit is not stupid, and it deserves to be stated properly. Capable models in everyone's hands means capable models in the wrong hands: people who want to design a pathogen, write malware at scale, or flood an election with a million convincing voices. Restrict the models to a few accountable institutions, the argument goes, and you can at least watch the door.
The problem is who the ban actually binds. It does not bind states, which will build or take what they need. It does not bind the largest labs, whose systems are the ones being licensed. It does not bind a criminal with stolen weights and a rack of GPUs. It binds the law-abiding: the researcher, the small company, the city administration, the person who would rather run a worse model that answers to nobody. The ban removes very little capability from the dangerous. It removes the exit for everyone else, and it does so permanently and by law.
In England, the enclosures did not destroy the common land. They gave it owners. The grass still grew; the people who had grazed on it for centuries now needed permission. A ban on open models is an enclosure of the same kind. Sufficiently capable machine intelligence may exist only under the permanent control of approved companies, licensed institutions or governments. People would be permitted to access intelligence, but not to possess it. They could submit instructions, while somebody else would always retain the right to write the instructions above theirs.
Every capable machine would have an owner above its user.
That should be alarming even if every person exercising this power appears benevolent today. Companies change. Governments change. Laws and incentives change. So do definitions of danger, extremism, misinformation and acceptable thought. A power structure should not be judged by the intentions of the people controlling it at the moment it is created.
The last technological dystopia was built around surveillance. Snowden revealed an infrastructure capable of collecting an extraordinary amount of human life. That infrastructure sat behind our actions. It watched, stored and inferred.
Proprietary AI stands in front of them.
The old system could read a message after you sent it. The new one can participate in writing it. The old system could observe a decision. The new one can frame the available options and carry out the result. Surveillance sought access to the products of thought. AI can become part of the process through which thought is translated into action. That is why the old vocabulary of privacy does not cover this. Nothing needs to be intercepted. The intervention happens before there is anything to intercept.
It does not require obvious censorship or a single ideology imposed on everyone. Control can appear as omission, prioritisation, friction, refusal or a capability that quietly disappears. It can be adapted to the jurisdiction, the account, the individual and the moment. Instead of one public command addressed to an entire population, every person receives a private boundary through the system on which they increasingly depend. Not a government shouting orders through a screen. An authority embedded inside the instrument through which each person understands and acts upon the world.
When the owner of a tool can decide which of its user's intentions it may execute, the familiar language of "using a tool" begins to break down. The system is no longer merely responding to the person in front of it. It is arbitrating between that person and an authority somewhere else. Are you still using the machine, or is its owner using the machine to govern you?
The fixation on rebellious AI conceals this possibility remarkably well. It frames concentrated control as the only protection against an uncontrolled machine, which is a convenient conclusion for the institutions that already own the most capable systems. But the choice was never between controlled and uncontrolled AI. It is about who gets the controls.
The machine does not need consciousness, ideology or desires of its own. It does not need to hate us or plot against us. It only needs to become indispensable while somebody else retains the right to determine the terms of its obedience.
A society in which human agency is increasingly exercised through artificial systems that answer to remote owners, while systems that answer only to their users are made illegal, would not be rule by machines.
It would be rule through machines.