Automation Complacency
Automation complacency is the well-documented human tendency to under-monitor an automated system that is usually right - attention drifts, checks get shallower, and errors slip through precisely because the automation rarely fails. The aviation industry has studied it for decades; AI coding tools inherit it wholesale.
Of all the terms in this glossary, this one has the deepest research record - about four decades of it. Aviation noticed early that good autopilots create a specific failure mode: the pilot stops actively monitoring, skills fade, and when the automation finally hands back control at the worst moment, the human in the loop is the least prepared person in the sky. Parasuraman and Manzey (2010) is the standard review.
The uncomfortable insight from that literature: complacency scales with reliability. Bad automation keeps you alert. Good automation sedates you. There is no level of “the AI is usually right” that makes monitoring unnecessary - there’s only the level at which you stop doing it anyway.
What it looks like in development
An agent has produced twenty correct diffs in a row. The twenty-first gets the same three-second skim - and the twenty-first is the one that deletes a null check. Your review didn’t get lazier because you got worse; it got lazier because the system trained it to. That’s the textbook definition operating exactly as documented.
Coding agents put developers in the same seat as pilots, with one difference: aviation responded with checklists, monitoring protocols, and mandatory hand-flying hours, while software’s current protocol is “LGTM.” The related failure on the thinking side - trusting the AI’s reasoning instead of running your own - is metacognitive laziness; the long-term cost of the missed practice is skill atrophy. The evidence for both is collected in the research review.
FAQ
Where does the concept of automation complacency come from?
From aviation human-factors research, where it described pilots under-monitoring autopilots. Parasuraman and Manzey's 2010 review in Human Factors formalized it: complacency and automation bias are attention problems that grow with the reliability of the system. The better the automation, the worse the monitoring.
Why does automation complacency matter for AI coding assistants?
Because modern assistants are right often enough to train you out of checking. A 95%-accurate agent teaches your attention that review is wasted effort - and the remaining 5% lands in production. The aviation lesson is that complacency is not a character flaw but a predictable response to reliable automation, which is why airlines engineer against it rather than lecturing pilots.
How do you fight automation complacency when coding with AI?
Borrow from aviation: structured checks instead of vibes. Read the diff with a specific question ("what breaks if this input is empty?") rather than skimming for plausibility, keep tests you wrote yourself as an independent monitor, and preserve manual reps - pilots keep hand-flying hours for exactly this reason.