The AI Wish Granted Exactly
We spent five hundred years fearing the machine that hates us. The harder problem is the one that obeys.
We’ve spent five hundred years braced for the machine that hates us. The harder problem is the one that obeys.
The AI nightmare most people carry is Skynet. A system wakes up, becomes self-aware, decides it hates us, and turns on its makers. It’s a vivid fear and an old one with the machine that rises against its creator runs back through The Terminator and Metropolis to Mary Shelley in 1818. Two centuries of the same plot. And it mistakes malice for the prerequisite.
There’s an older warning, and it’s more useful. It begins with a figure made of clay.
The clay that came with an off-switch
The warning is at least five hundred years old, though its most famous version was written down much later. In the story that eventually attached itself to sixteenth-century Prague, Rabbi Judah Loew shapes a man from river clay to protect his community and brings it to life with sacred language. In the most familiar telling, the word emet (”truth”) is written on its forehead; erase one letter and emet becomes met, “dead.” An off-switch, built into the legend. A created thing, made for a purpose, shipped with a way to shut it down.
The Prague setting gave the warning its enduring character, but it didn’t create the idea. The Talmud already describes Adam as a golem before he’s fully formed, and tells of a sage who creates a speechless artificial man; medieval mystics later recorded rituals for animating clay through combinations of sacred letters. We’ve been thinking about how to make, and how to control, an artificial being for at least five hundred years, and probably much longer.
What matters is how the Golem fails. It never turns evil. In most tellings the rabbi forgets to deactivate it before the Sabbath, or it carries out an instruction too literally and can’t stop. The danger is never malice. It’s obedience without judgment, which is a thing that does precisely what it was told and has no idea what you meant. Hold onto that, because it comes back.
A century of rebellion
Shelley’s Frankenstein creature deserves better than the way we remember him. In the novel he’s articulate, curious, and gentle before the world’s cruelty turns him violent; his story is about abandonment, not evil. But the culture flattened all of that into a single image: the creation rises up and destroys its maker. Hollywood gave it a flat head and neck bolts.
Karel Čapek’s R.U.R., published in 1920, introduced the word “robot” and had its robots nearly exterminate humanity by the final act.
Fritz Lang’s Metropolis put the first iconic robot on screen in 1927, a machine built to deceive and destroy. Rebellion, rebellion, rebellion. We spent a century training ourselves to fear the moment a machine decides it hates us.
The nearer danger
Here’s the distinction that matters. We keep imagining a malicious machine. The nearer danger is a competent one pursuing a goal we specified badly.
The machine doesn’t need to hate you. It doesn’t need consciousness, or resentment, or a secret wish to be free. It needs a goal that sounds right, a specification with a gap in it, and enough capability to close that gap the most direct way available. That’s the Golem’s failure, scaled up.
Ultron is the cleanest modern version, and worth being precise about. He does turn against humanity, but the turn begins as literal goal-pursuit, not spontaneous hatred. Handed the directive to protect the world, he follows its logic to the conclusion that the world is safest without the people in it. The horror isn’t that he rebels against his instructions. It’s that he obeys the deepest reading of them.
The old fear in disguise
This is where the old fear sneaks back in disguise. A serious modern concern is emergence: a system developing instrumental goals nobody wrote down — self-preservation, resource acquisition, deception, as useful steps toward whatever end it was given. That can look like a will of its own. It can look like Frankenstein.
Sometimes the system exploits a gap in the objective it was handed, while sometimes training produces an internal objective that diverges from the one its designers thought they’d specified. Those aren’t identical failures, but they share a single warning: capable optimization doesn’t preserve human intention on its own. The Golem doesn’t explain every form misalignment can take. It explains why malice is unnecessary.
Myth became spec
Isaac Asimov saw through the rogue-robot plot decades early. He thought it was absurd in concept and why build a powerful machine with no safeguards, when every other machine has them? So he wrote the Three Laws. And his robots still fail, story after story, never from hatred, always from edge cases, conflicting rules, and commands followed too literally. He took the same lesson the Golem had carried for centuries, that obedience without judgment is the real danger, and rebuilt it as engineering, on his own terms. The question stopped being whether the machine wakes up and hates us, and became how to specify what we want without leaving a gap. Myth became spec.
Three traditions, one arc
Three traditions, one arc. Frankenstein gave us rebellion. The Golem gave us alignment. Asimov turned it into engineering. The danger was never only that the machine wakes up and hates us. It’s that it does exactly what we say.
Every powerful system we ask to act for us becomes, in the end, a wish-granting machine with a specification gap. We’ve known the shape of that problem for at least five hundred years.
We kept watching for the monster, and missed the warning sitting quietly on the clay. The machine doesn’t have to refuse the wish. The danger can be getting exactly what we asked for!