Stereotyping the work of aviation accident investigation boards (AIBs) may become impossible in light of presentations by several of their leaders at a recent global conference. More openness to data-mining methods pioneered by airlines and civil aviation authorities (CAAs) is apparent, and some AIBs already have extensive experience conducting analyses of their own safety data for predictive purposes. Leaders and investigators who had been reluctant to move in this direction now point to the influence of non-AIB initiatives with international reach, such as the Aviation Safety Information Analysis and Sharing program in the United States.
A few AIBs envision complementing their own safety-intelligence resources by partnering with CAAs for access to aggregated, de-identified airline data from voluntary reporting systems and/or flight data monitoring, although such data formerly had been considered incompatible with AIB methods and missions (ASW, 5/11, p. 18; ASW, 11/12, p. 31).
Completing accident investigations while venturing into predictive data analysis and risk mitigation was the theme of the joint International Investigative Issues Conference of state AIB leaders and the International Society of Air Safety Investigators ISASI 2012 seminar in August.
Deborah A.P. Hersman, chairman of the U.S. National Transportation Safety Board (NTSB), opened the conference saying: “Now I know that this might be hard for you all to hear, but some even suggest that with today’s sophisticated data and tools, accident investigations may become a thing of the past. However, I would propose that the reality is that forensic investigation is the foundational tool not only to be reactive, but to be more predictive. … In almost every accident, we do see precursors, data that could have been used to understand breakdowns in safety margins and predict accidents. But in real time, with thousands of flights, figuring out what the data can tell you, and how it might combine with other factors in the operating environment … can be as much of an art as science. … [AIBs] must use all the tools available — retaining the ‘tin-kicking,’ but also enhancing laboratory equipment and taking advantage of tools that mine data to map trends and hot spots — so we can move from reactive to predictive.”
Comparing data even from minor incidents to the experience of entire fleets enables AIBs to include predictive elements when issuing safety recommendations, she said.
“It’s not beneficial for any AIB to look at a single accident now in a vacuum,” Hersman said. “We have tremendous data sources out there waiting to be mined. … If we’re not understanding how this [accident] relates in the context, we’re not really helping to advance things. … And so [AIBs] have to be part of that feedback loop. Sometimes one of the challenges … is not just siloing within organizations, but it’s siloing organizations within the [safety management system] process.”
NTSB by law conducts special studies and investigations apart from investigating aircraft accidents, added Joseph Kolly, director, and Loren Groff, national resource specialist, both in NTSB’s Office of Research and Engineering. These have similarities to the newer interest in predictive data analysis. Data sources have comprised existing databases and targeted collection of new data, including aggregate data from nominal/non-accident flight operations for understanding safety issues, for comparisons to accidents and for predictive uses. “Independent investigation authorities typically have access to a wealth of detailed information regarding the circumstances surrounding safety management failures and hazardous events,” they said.
Facing Canadian Realities
Noting the latest 10-year record of one fatal accident in Canada involving large commercial jets, Wendy Tadros, chair of the Transportation Safety Board (TSB) of Canada, rhetorically asked, “With the numbers in decline, where does that leave us — where does that leave you, as investigators? Will the numbers ever reach and stay at zero? Will we have to close up shop and go home? The reality is, in this complex world … many of you will be investigating large aircraft accidents in the developing world as accredited representatives.”
Inside Canada, one of TSB’s most important tasks is to prudently invest resources only in investigations of accidents that promise to yield valuable knowledge, and then multiply the value of results by finding patterns through predictive analysis of data. When it comes to providing the hard evidence required to push government and industry toward AIB-recommended changes, statistics are the most persuasive, Tadros said, noting, “What can start off as a ‘weak signal’ in one occurrence, or in several occurrences … may be a symptom — a sign of greater trouble — down the road. … I am not trying to talk about predicting the future; rather I’m talking about studying the details, recognizing those underlying factors, the ones that maybe haven’t become full-fledged causes yet, but which are nonetheless important.”
Canada’s smaller turboprop and piston-engine aircraft — widely used in commercial operations such as on-demand flights, as well as private flying — today are involved in more than 90 percent of accidents and 90 percent of fatalities, she said. TSB has concluded, given their predictive value, that safety management systems (SMS) and flight data recorders should be implemented by small operators in addition to large operators.
“Aircraft accident safety boards have a very important role to play in the evolution of safety management from reactive to predictive,” said Michael Cunningham, Air Branch Atlantic regional manager, TSB. “Some systems may even be capable of evolving from reactive to predictive abilities as a result of research and development by experts in organizational management, fueled with the results of comprehensive safety board investigations.”
Therefore, TSB has placed major emphasis on developing investigators’ expertise in SMS through the study of Transport Canada regulations and guidance; a TSB course and annual workshops; a TSB on-site SMS review guide; presentations by industry SMS managers; and sharing examples of problems identified by TSB, such as airline management taking inappropriate punitive actions against employees.
AIBs play a key role in objectively informing governments about the current and future risks to the nation’s aviation industry, said Stuart Godley, manager, research investigations and data analysis, Australian Transport Safety Bureau (ATSB). “Like airlines, nations rely on the collective wisdom of a combination of proactive initiatives, in addition to the reactive investigations that we do,” he said. ATSB integrates predictive research primarily by using its internally generated data. Specifically, ATSB has been performing quarterly trend analysis for about two years, comparing current findings to five-year averages.
Predictive analysis has become “an integral way of conducting business as an investigator [agency] by guiding the best way that we can use our limited resources,” Godley said. “Importantly, another driver is the prospect of discovering … data that could have been used to highlight an emerging risk before a catastrophic accident happened. But we haven’t actually used that data to do that.”
AIBs often keep a nation’s official accident and incident database and also may have a non-public database of details from accident/incident investigations, enabling trend monitoring, decision making, risk rating of occurrences, analysis of findings across investigations, and proactive analysis of safety deficiencies.
“Looking [annually at about 7,400] non-investigated occurrences provides a very valuable … visibility of safety not always available in that small subset [of about 100 in aviation] that we investigate,” Godley said. “We’re looking at everything to see if there’s any subtle changes that may point to something bigger.
“[We’re] then using statistics, seeing if the last quarter is greater than or less than, say, two standard deviations. … When that happens, you can call that a ‘hard alert.’ … Sometimes there’s no real trend or just a bit of random noise, and sometimes [it’s] quite clearly coming from one area.” The ATSB response to a hard alert may include designating one investigator to closely monitor all related occurrences day by day and to develop sensitivity to the details of each.
An event risk classification system published for airlines by Aviation Risk Management Solutions — and free for anyone to adopt, Godley said — helps AIBs to make one-time investigation decisions and to analyze multiple related occurrences. This results in a risk score rating matrix (Figure 1).
The system has been used, for example, to establish ratings for the risk of bird strikes based on factors such as actual strike history, whether increases in data reflect changes in strikes or changes in reporting, and the types and number of birds involved in strikes.
Sorting the risk matrix by airports has influenced the government to act by showing that “the airports with the greatest risk, such as Townsville and Alice Springs, [have a higher risk] because aircraft are actually having bird strikes of multiple … medium- and large-sized birds as opposed to just having [strikes involving] one bird or smaller birds,” he said.
AIB Nigeria’s FOQA Focus
Muhtar Usman, commissioner/CEO of Accident Investigation Bureau Nigeria and a captain, and Mike Poole, director, safety and strategy, CAE Flightscape, told the conference that in many states, neither AIBs nor CAAs “have embraced the use of flight data for proactive safety measures.” However, a strong motivation for such programs is the International Civil Aviation Organization’s (ICAO’s) latest emphasis for states to investigate serious incidents and also to collect data under state safety programs.
Usman said that the Nigerian government, for example, in late 2012 was scheduled to establish its own flight data analysis capability under a plan to require operators of Nigerian-registered aircraft to provide flight data to AIB Nigeria for the state safety program. He said that “the data will be kept confidential and used to develop safety action in the form of safety recommendations; it will not be used for punitive measures and it is not intended in any way to replace an operator’s internal FOQA [flight operational quality assurance] program.” The objectives include focusing on broad issues that transcend airline boundaries and/or aircraft type, and identifying the most problematic airports and events nationwide.
Other AIBs also have been adding infrastructure that enables broader and deeper safety data analysis suitable for predictive purposes. For example, memorandums of understanding and cooperation agreements that include provisions for safety data exchange have been signed by Aviation Safety Council (ASC) Taiwan with other AIBs, government agencies and the academic research community, said Wen Lin (Michael) Guan, director of the investigation laboratory at ASC.
“At ASC, we have built up an occurrence information management system,” Guan said. “We know [beforehand even] the type of recorder installed on the aircraft. We also know the number of [flight data recorder parameters].” The system also organizes nearly all resources and information during an investigation and remains a resource for subsequent larger-scale safety analyses.
AIB laboratories today must capture and manage data associated with accidents in more sophisticated ways than before, agreed Sylvie Dionne, manager, materials analysis and structures operational services branch, TSB. “You’re going to learn a lot from previous cases and from ongoing monitoring of what’s going on, so you need to have some kind of database, I would say,” she said.
In Japan, the independence of the Japan Transport Safety Board (JTSB) from the Japan Civil Aviation Bureau, the nation’s regulator and air traffic service provider, preserves neutrality and transparency in investigations but also may affect efforts to adopt predictive safety analysis, according to Yuji Yanagisawa, deputy investigator general for aircraft accidents at JTSB. “These two organizations have a different role to play, but are working toward the same goal of improving the air safety — especially now, shifting to the state safety program and SMS environment, where aircraft accident investigation and regulatory enforcement are parts of the component elements,” he said.
Europe Settles AIB Debate
Europe largely has resolved complex disputes over relationships between the European Aviation Safety Agency (EASA) and the region’s AIBs, said John Vincent, deputy director for strategic safety, EASA. How to aggregate and use the region’s newest data source, airlines’ voluntary safety reporting systems, is still under discussion.
“We had a long period of debate as to what EASA’s role was in relation to accident investigation,” Vincent said. “We hope that’s resolved now because we have … Regulation 996,1 which came into being in 2010 and establishes in European law the position that EASA has relative to the air accident investigators. I’m very glad to say that we work together [with AIBs and CAAs] as a team, the objective being the continuous improvement of aviation safety. It’s taken some time to build and develop that relationship; in some quarters, it’s still growing and developing and maturing.” A high level of interaction and interdependency, including for data uses, is expected, he said. In 2012, EASA also began a process under Regulation 996 to establish one central European database for AIB and CAA safety recommendations.
In Brazil, after a period of SMS-related adjustments to their relationship, the AIB —Centro de Investigação e Prevenção de Acidentes (CENIPA) — and the CAA have begun a new phase in which CENIPA is the national center for SMS training and also sees opportunities to pursue predictive data analysis, said Col. Fernando Camargo, former deputy chief of CENIPA. He described the adjustments as a positive step in resolving difficulties of interagency cooperation during investigations and in CAA acceptance of its safety recommendations.
Expectations of ICAO’s Universal Safety Oversight Audit Program also have motivated sharing of one database by the CAA and CENIPA for information such as final reports of accident/incident investigations and voluntary non-punitive reports from airline personnel, Camargo said.
U.K. Iterative Investigations
Philip Sleight, principal inspector, U.K. Air Accidents Investigation Branch (AAIB), asked, “What is the goal of … a no-blame accident investigation? I would say actually the goal is to identify the safety issues early so that actions can take place to prevent recurrence.” The agency’s holistic model — essentially, iterative methods and free-flowing, cross-domain communication among individuals and groups participating in an investigation — has proved to be consistent with this goal, effective for safety mitigations prior to issuance of an accident report, and compatible with current proactive/predictive concepts of safety data analysis, he said.
Yet one challenge AAIB has recognized is that, whatever proactive/predictive initiatives may be desirable, the data demands of accident investigations alone become extremely difficult as complexity of aircraft and methods continue to increase. “Certainly at the AAIB, we would actually struggle with storing all of this information [and] in such a way that it is easily retrieved,” Sleight said. “As we were doing the investigation of the [British Airways Boeing 777 runway-undershoot accident www.aaib.gov.uk/cms_resources.cfm?file=/1-2010%20G-YMMM.pdf] at Heathrow — with vast amounts of information that were coming in at a very rapid rate — we were still exploring solutions to enable all parties to the investigation to have secure access to the information, but also in a timely manner without compromising confidentiality.”
- The title is Regulation (EU) No 996/2010 of the European Parliament and of the Council of 20 October 2010 on the investigation and prevention of accidents and incidents in civil aviation and repealing Directive 94/56/EC.