[masraum]

So often at work after an incident, mgmt is out looking for someone to place blame on. And yes, there's often a final triggering event that "causes" an incident, but in complex systems, there's usually a perfect storm of issues and failures before an ultimate failure, so there really is no single person that was the "root cause". The real root cause is more like - system A was in a bad state, system B had an error condition, system C had been disabled, and finally, Bob hit the button in system D that caused the whole thing to come crashing down. But if the first 3 conditions in systems A, B, and C hadn't existed, then Bob hitting the button wouldn't have been a problem or would have been a much different, smaller (non-catastrophic) issue.

https://how.complexsystems.fail/
(note, the following is not the full text of the link, but is excerpts of each of the points)

How Complex Systems Fail

(Being a Short Treatise on the Nature of Failure; How Failure is Evaluated; How Failure is Attributed to Proximate Cause; and the Resulting New Understanding of Patient Safety)
Richard I. Cook, MD
Cognitive Technologies Labratory
University of Chicago

1 Complex systems are intrinsically hazardous systems.
All of the interesting systems (e.g. transportation, healthcare, power generation) are inherently and unavoidably hazardous by the own nature.

2 Complex systems are heavily and successfully defended against failure
The high consequences of failure lead over time to the construction of multiple layers of defense against failure.

3 Catastrophe requires multiple failures – single point failures are not enough.
The array of defenses works. System operations are generally successful. Overt catastrophic failure occurs when small, apparently innocuous failures join to create opportunity for a systemic accident.

4 Complex systems contain changing mixtures of failures latent within them.
The complexity of these systems makes it impossible for them to run without multiple flaws being present.

5 Complex systems run in degraded mode.
A corollary to the preceding point is that complex systems run as broken systems. The system continues to function because it contains so many redundancies and because people can make it function, despite the presence of many flaws.

6 Catastrophe is always just around the corner.
Complex systems possess potential for catastrophic failure.

7 Post-accident attribution to a ‘root cause’ is fundamentally wrong.
Because overt failure requires multiple faults, there is no isolated ‘cause’ of an accident. There are multiple contributors to accidents. Each of these is necessarily insufficient in itself to create an accident. Only jointly are these causes sufficient to create an accident.

8 Hindsight biases post-accident assessments of human performance.
Knowledge of the outcome makes it seem that events leading to the outcome should have appeared more salient to practitioners at the time than was actually the case.

9 Human operators have dual roles: as producers & as defenders against failure.
The system practitioners operate the system in order to produce its desired product and also work to forestall accidents.

10 All practitioner actions are gambles.
After accidents, the overt failure often appears to have been inevitable and the practitioner’s actions as blunders or deliberate willful disregard of certain impending failure.

11 Actions at the sharp end resolve all ambiguity.
Organizations are ambiguous, often intentionally, about the relationship between production targets, efficient use of resources, economy and costs of operations, and acceptable risks of low and high consequence accidents.

12 Human practitioners are the adaptable element of complex systems.
Practitioners and first line management actively adapt the system to maximize production and minimize accidents.

13 Human expertise in complex systems is constantly changing
Complex systems require substantial human expertise in their operation and management.

14 Change introduces new forms of failure.
The low rate of overt accidents in reliable systems may encourage changes, especially the use of new technology, to decrease the number of low consequence but high frequency failures. These changes maybe actually create opportunities for new, low frequency but high consequence failures.

15 Views of ‘cause’ limit the effectiveness of defenses against future events.
Post-accident remedies for “human error” are usually predicated on obstructing activities that can “cause” accidents.

16 Safety is a characteristic of systems and not of their components
Safety is an emergent property of systems; it does not reside in a person, device or department of an organization or system.

17 People continuously create safety.
Failure free operations are the result of activities of people who work to keep the system within the boundaries of tolerable performance.

18 Failure free operations require experience with failure.
Recognizing hazard and successfully manipulating system operations to remain inside the tolerable performance boundaries requires intimate contact with failure.