The model took the blame for a human decision

Meta description: Blame moves to the model because the model cannot answer back — but the framing, the input, and the review were all human. More on his work: his consulting practice.

Miklos Roth

The postmortem was thorough about the model. Its confidence scores, its training data, its known limitations — slide after slide of careful technical analysis. I read these documents professionally now, and I have learned to look for the missing slide: the moment a person read the model’s output and sent it onward. That moment was a decision. It had a decider, a context, an alternative. In the postmortem, it appears as “the output was used,” in the passive voice, assigned to no one. The model cannot defend itself, and in the reviews I read, it does not need to. It absorbs everything.

[AWAITING MIKLOS — scene: a postmortem whose first slide is the model and whose absent slide is the decision to accept the output]

How does a postmortem start when blame has already moved?

The starting question reveals everything. A postmortem whose blame has already moved asks: why did the model get it wrong? The investigation that follows is real and useful — and it is bounded by the question. I have watched reviews spend weeks on the model’s failure and minutes on the human’s acceptance, though the acceptance was the last point where the harm was preventable. The order of slides is the order of absolution: by the time the review reaches the human in the chain, the causal story is already written, and the human appears as the person unfortunate enough to be holding the output when it failed.

What human decision disappears in that telling?

The line I draw: the model is responsible for nothing; the system of people around it is responsible for everything. A bad result is not automatically the LLM’s fault. The model has produced an answer to the question it received, not necessarily to the question the business needed to ask. If the organization has not defined what acceptable output is, or who is responsible for catching mistakes, the apparent failure of the tool is a failure of specification and review.

The disappearing decision is acceptance — the read, the judgment, the send. Before AI assistance, acceptance was the job: a professional examined the material and decided it was fit. The model inserted itself upstream of that decision, and the decision did not vanish; it shrank. I have watched acceptance compress from an examination to a scan to a click, while remaining, formally, the decision on which the harm depended. The postmortem that blames the model treats the compressed acceptance as a formality. It was not a formality when the same person accepted good outputs for a year and was credited with good judgment.

Why do disclaimers stabilize the shift?

The false explanation is communication: outputs need clearer disclaimers, users need better warnings about model limitations. I have read the disclaimers; they are present, prominent, and useless in the way that matters. A disclaimer transfers responsibility in text while the workflow transfers it in fact — the workflow gives the human seconds per output and calls it review. The disclaimer then functions as an alibi for the organization: the user was warned. Everyone is covered, the disclaimer is cited in the postmortem, and the structural fact — that the human’s acceptance was designed to be ceremonial — survives to the next incident.

What pattern marks scapegoating versus a real model fault?

The distinction I use is counterfactual. With a real model fault, a different model would have prevented the harm — the human’s process was sound and the tool failed it. With scapegoating, a different human process would have prevented it — the tool behaved as tools behave, and the acceptance failed. I apply this to incident reviews and find the second kind carrying the first kind’s blame with remarkable consistency. The finding is uncomfortable, because the second kind implicates design decisions made by people senior to everyone in the postmortem room.

In a postmortem I would separate four candidates before assigning cause: the input, the process, the tool, and the standard itself. Quite often, human review reveals changes that are necessary before a result is usable — I regard that as part of the operating process rather than an unfortunate add-on. The organizations that learn are the ones that treat each disappointing result as evidence about their own system, not as a verdict on the technology.

[AWAITING MIKLOS — scene: second scene]

What question returns the decision to a person?

Blame conversations ask what went wrong, and the model is always available as an answer. The question that interrupts the pattern asks where the last human decision was, and what that decision would have needed to go differently. I have watched that question redirect reviews from the model’s internals to the decision’s conditions — the time given, the authority held, the escalation path that did or did not exist. The model stops being the protagonist. The story gets shorter and harder to file.

The question that relocates the blame: which of the four — input, process, tool, or standard — actually failed, and what does the postmortem change as a result?

PR excerpt: [PR — awaiting Miklos material; cut 80–120 words from his paragraphs once available.]

© Copyright babaszobafertotlenites