Essay
Why Bainbridge's 1983 paper is the sharpest lens on AI deployment in 2026.
In 1983, Lisanne Bainbridge published a four-page paper called Ironies of Automation. It was written about industrial control rooms โ operators supervising chemical plants and power stations. It is the most useful document I know for understanding why AI tools fail inside organisations today.
Her central irony: the more you automate a task, the less capable humans become at supervising the automation. The system handles the routine cases, so operators lose the practice that built their judgment. Then the automation hits an edge case โ precisely the situation it wasn't designed for โ and hands control back to a human whose skills have quietly atrophied. Automation doesn't remove the human from the loop. It removes the human's readiness and leaves them in the loop anyway, at the worst possible moment.
Every clause maps directly onto LLM deployment.
Irony 1: The designer's errors become the system's errors. Bainbridge noted that automation is designed by people who assume the human is unreliable โ yet the designer is also human, and their assumptions get baked in permanently. In AI terms: a prompt, a fine-tune, a workflow encodes its author's model of the task. The model then applies that assumption to every case, including the ones the author never imagined. Confidently.
Irony 2: The human keeps the tasks the designer couldn't automate โ which are the hardest ones. The LLM drafts the routine proposal; the human handles the weird client, the ambiguous contract, the case that doesn't fit the template. We've automated the practice reps and kept the finals.
Irony 3: Monitoring is a terrible job for humans. People cannot sustain vigilance over a system that is right 95% of the time. They stop checking. This is the operational reality behind every "human in the loop" checkbox: the loop exists on the org chart and nowhere else. An LLM output that is usually right trains its reviewer to approve without reading โ which is exactly when the 5% ships.
Irony 4: Skill decays without use. The junior analyst who never writes the first draft never develops the judgment to evaluate one. In two years the organisation has no one who can tell a good output from a plausible one. The automation has consumed its own supervision.
Bainbridge's ironies aren't an argument against automation. They're a design specification. Systems that survive them share properties:
Make failures loud, not smooth. A system that guesses gracefully at edge cases is optimised to hide exactly what humans need to see. Explicit exception queues beat plausible interpolation. The most dangerous output an AI system can produce is a wrong answer formatted like a right one.
Give the model proposal rights, not commit rights. If a deterministic layer checks every model output before it takes effect, the vigilance problem shrinks from "check everything" to "review the exceptions." Machines are good at tireless checking; humans are good at judgment on a bounded set. Assign accordingly.
Preserve the practice that builds supervisors. If a workflow matters, someone must retain the ability to do it unassisted. This is a staffing and process decision, not a tooling one โ and it's the one organisations skip first.
Instrument everything. Bainbridge's operators couldn't see inside the automation. Yours shouldn't have that excuse. If leadership can't see exactly what the system decided and why, they will discover its failure mode from a customer.
The final irony is ours: forty-three years of warning, freely available, four pages long โ and the current wave of AI deployment is recapitulating every failure mode on schedule. Tools that demo brilliantly, degrade silently, and leave behind teams less capable of catching the degradation than before the tool arrived.
The pattern is fixable. But the fix is architectural, not promptual. You don't solve Bainbridge with a better model. You solve it by deciding, precisely, what the model is allowed to own โ and building the substrate that owns everything else.
Bainbridge, L. (1983). Ironies of Automation. Automatica, 19(6), 775โ779. For the pattern in practice, see Where Truth Runs Out.