When Software Goes Wrong - Engaging Your Imagination

In this seventh and final episode about software failures in safety-critical systems, I talk with Dr Emma Taylor about a number of incidents, from the railway, from medicine, from shipping, and from aircraft.  Our purpose is to encourage you to be creative when you start to think about reasonably foreseeable scenarios—what could possibly go wrong with your complex software-based system.  And what you can do to help design out errors before they cause an incident.

Listen to the podcast

Topics in this episode include:

• Emma describes ways to look beyond a railway specific risk analysis to identify reasonably foreseeable scenarios.  There may not be enough railway incidents to learn from, so we need to look elsewhere. [1:42]
• Emma makes the point that we must look to the past to identify what could happen in the future.  And the importance of using your own scenarios when planning for digital safety and security.  [2:48]
• Emma talks about the concepts she will expand on to help you start your own thinking processes.  [3:37]
• Emma talks about common themes, some considerations that might be relevant to your software development.  [4:36]
• Emma talks about the continued use of old hardware and software and their safety weaknesses.  She emphasises the importance of reporting and recording of faults that happen in software-based systems.  [9:20]
• Emma talks about the need to look to other sectors and industries to learn from the potential impact of software.  The importance of thinking about recovery scenarios should the worst possible incident happen.  [8:24]
• Emma talks about how to carry out an effective risk assessment for digital systems, and about the need to be able to let the right people into your system, but keeping out the unwanted ones.  [12:03]
• Emma talks about a 2019 incident with the Class 700 and 717 trains following the application of a software patch, that had unintended consequences.  [13:18]
• Emma makes it clear that you don't need to ask complex questions to capture reasonably foreseeable scenarios for complex software-based systems.  [15:40]
• Ant touches on the apparently simple software patch on the 737-8 Max aircraft that led to hundreds of fatalities  [15:59
• Emma talks about software and cyber security using a house as a model—with doors and windows that you may, or may not, want to allow people to come through.  [16:43]
• Close [18:18]

Resources and other sources mentioned in this episode:

• Online article quoted about the Maersk malware incident

Boeing 737-8 Max aircraft reports 

• How the Boeing 737 Max Disaster Looks to a Software Developer (a personal view) 
• Federal Aviation Authority Updates on Boeing 737 MAX
• Federal Aviation Authority—Boeing 737 MAX Reading Room (for the real detail geeks)
• Boeing's own 737 MAX Updates web page—includes links to the plane's validation process 

Class 700 and 717 incident report

• ORR: ' Report following railway power disruption on 9th August 2019'  (PDF download)

Other related resources:

• LHSBR Infrastructure Asset Integrity section
• LHSBR Rolling Stock Asset Integrity section
• The digital bits of a system podcast
• The V-model on Geeks for Geeks.org
• The V-model for humans on Wikipedia