Hindsight Bias and Fundamental Attribution Error

Related Articles:
Rethink Safety
The Leader

"Why did you do this? You should have done that." Most of us (including me) are guilty of similar thoughts or words. 

This is a classic example of hindsight bias (knew-it-all-along effect). 

Accident investigators already know the accident outcomes and sequence of events. Hence, they tend to be biased and focused on significant data points leading to the accident. Instead of blaming individual actions, it would be more meaningful to understand why individuals took or did not take appropriate actions based on the information available and context facing them leading to the accident. Recommendations should focus on the changes and improvements required to help individuals make better decisions in future to prevent accidents. Of course, the underlying assumption is that humans do not possess malicious intent.

Another concept is that of fundamental attribution error. The observer overvalues his own point of view and undervalues the actual situational context faced by the actor being observed. If the roles of the observer and actor are swapped, the same fundamental attribution error is likely to occur. 

It is almost impossible to eliminate hindsight bias and fundmental attribution error. The takeaway is to  focus on systems rather than individuals.  Improving the design of the systems (including technical, social and sociotechnical systems) is certainly more productive than focusing on blame. For instance, performance measurement systems shape and drive the behaviors of employees in enterprises. Who is to blame when undesired outcomes occur? 

Again, even if the system in place is flawless (is there one?), human ingenuity, the very reason that differentiates humans from other living things on Earth, more often than not, is the very same reason that collapses the systems we built. Seems like "People" are stickier problems than "Technology". Leadership may be the best solution available.  

Thanks very much for reading. 

Rethink Safety

Related Article: Hindsight Bias and Fundamental Attribution Error

Today's the last class for "16.863/ESD.863 System Safety". This class is taught by Professor Nancy Leveson, Professor of Aeronautics and Astronautics, and Professor of Engineering Systems at MIT. She is well recognized for her work in system and software safety. More of her work can be found at here.

In the aftermath of accidents, reports and legal proceedings tend to focus on assigning blame to individuals whose actions led to the mishaps. Assigning blame is easy,  but does very little to prevent future accidents from happening. My view on safety has changed over the course of the class. Specifically, it shifted from focusing on individual responsibility and errors to viewing safety as a control problem. Also, it reaffirmed my view that quantifying safety with probablisitic numbers is not meaningful. 10^-9 for software coding errors? How did the number come about?

STAMP (System-Theoretic Accident Modeling and Processes) and STPA (System-Theoretic Process Analysis) are powerful techniques developed by Professor Leveson to perform accident analysis and hazard analysis. The techniques  (Leveson, 2012) are not perfect, but certainly better than unrealistic probabilistic estimations. These techniques can be applied to sociotechnical systems, and a wide variety of industry applications including aerospace, nuclear, financial, chemical, oil and gas, and so on. More details, examples, and applications of STAMP/STPA can be found in the book "Engineering a Safer World"and here.

The just culture movement (Dekker, 2008) promotes learning from mistakes rather than blaming and punishing people. Being blame-free does not equate to being accountability-free. On the contrary, just culture promotes the active involvement of people to help build better and safer systems to work in or with. The key is to see accountability as forward-looking instead of backward looking. Backward looking includes the acts of blaming and punishing which does not really help to reduce the reoccurrence of similar accidents. The forward-looking mindset concentrates on making improvements to the system to prevent similar accidents from happening again. 

Actions of improvement speak louder than words of blame. Punishment serves as deterrence but does little to prevent future accidents if the system is inherently deficient. It is easier to design safer systems than to change people. Be safe than sorry!

As part of my class project, I came up with a safety control structure for the Traffic Collision Avoidance System (TCAS). Professor Leveson liked it and I'm sharing it here.

Thanks very much for reading.

Works Cited

Leveson, N. G. (2012). Engineering a Safer World. MA: The MIT Press.

Dekker, S. (2008). Just culture: who gets to draw the line? Cogn Tech Work.

Teo, K. S. (2012). 16.863/ESD.863 System Safety Class Project: STPA for TCAS. MIT, ESD.