One warning may be missed. A second check may use the same faulty information as the first. A third may be unavailable during a busy shift. In a 2000 paper, James Reason illustrated how defects in several safety layers can line up using slices of Swiss cheese.


Reason distinguished immediate mistakes from conditions that make them easier to commit or harder to catch. The latter can include poor design, lack of training or pressures that leave people little room to check their work. A failure investigation that starts and ends with the person closest to the harm can miss those conditions.
For an AI system, the analogy suggests a way to examine a safety plan. An automated check, a human review and a way to undo an action can each help. Their protection depends on whether they can catch different failures in the conditions where the system actually runs.
