Citing lapses in information gathering and flaws in accident analysis, the Transportation Safety Board of Canada (TSB) has completed its review of the Australian Transport Safety Bureau’s (ATSB’s) accident investigation methods by issuing 14 safety recommendations but also pledging to incorporate some of ATSB’s best practices into its own operations.
The review, requested by the ATSB in 2013, examined the ATSB’s investigations of three accidents but focused on the Nov. 18, 2009, ditching of an Israel Aircraft Industries Westwind 1124A near Norfolk Island during an emergency medical services flight. The TSB identified several problems, including poor data collection that hindered the ATSB’s analysis of several safety issues, among them crew fatigue, fuel management, and company and regulatory oversight.
Overall, the TSB said, public reaction to the Norfolk Island report “clearly demonstrated that it did not address key issues in the way the Australian aviation industry and members of the public expected.”
Nevertheless, despite the criticisms, the TSB review concluded that, in general, the Australian agency, like its Canadian counterpart, used investigative methodologies that “met or exceeded the intent and spirit of those prescribed” by the International Civil Aviation Organization (ICAO). The review recommended that some of those methodologies be adapted for use in the TSB’s investigations of accidents in Canada.
After receiving the TSB’s review, the ATSB said it was reopening the Norfolk Island investigation and “methodically and carefully” examining the TSB’s findings and recommendations “with the aim of ongoing improvements to the future work of the ATSB.”
A new team of investigators will review the original process, including its associated report, “in light of any fresh evidence and relevant points raised in the TSB review and other recent aviation reviews,” the ATSB said.
Original Report
In its original final report on the ditching, released in 2012,1 the ATSB said that the flight crew’s incomplete preflight and en route planning and their late assessment of landing conditions left them — after two missed approaches at the Norfolk Island Airport — with a choice of making a third approach at night in deteriorating weather conditions or ditching the airplane before fuel was depleted (ASW, 10/12).
They chose to ditch, and although the airplane broke apart after hitting the water and then sank, all six occupants escaped. Two suffered serious injuries, and the others either received minor injuries or were uninjured.
The ATSB’s conclusions were sharply criticized by some voices in the Australian news media and by the Australian Senate’s Rural and Regional Affairs and Transport References Committee, which accused the agency of defending its conclusions “without … a solid evidentiary base.”
The Senate committee complained that the ATSB had “repeatedly deflected suggestions that significant deficiencies with both the operator … and [the Australian Civil Aviation Safety Authority (CASA)] … contributed to the accident. The committee takes a different view.”
In the same report, the Senate committee accused the ATSB of allowing its investigative processes to become “deficient.”
‘Potential Shortcomings’
Although the TSB did not reinvestigate the Norfolk Island accident, it reviewed the methods used in the original investigation and singled out five issues that it said the ATSB’s report did not adequately address:
- Regulations that allowed the flight to proceed without the selection of an alternate airport in case of instrument meteorological conditions at the intended destination;
- “The extent to which the flight-planning and fuel-management deficiencies observed in this occurrence extended throughout the company”;
- The quality of CASA’s oversight of the operator, Pel-Air;
- The potential for flight crew fatigue and the quality of Pel-Air’s fatigue management efforts (the TSB said that the ATSB did not use a specific tool to collect and analyze data on fatigue); and,
- Issues involving the egress of the airplane’s occupants after the ditching.
In most of those areas, the original accident report’s analysis was incomplete because insufficient information was collected, the TSB said.
Overall, the review “revealed lapses in the application of the ATSB methodology with respect to the collection of factual information and a lack of an iterative approach to analysis,” the TSB said. “The review also identified potential shortcomings in ATSB processes, whereby errors and flawed analysis stemming from the poor application of existing processes were not mitigated.”
The TSB’s report cited an early failure to resolve a misunderstanding about the investigative responsibilities of the ATSB and CASA — a misunderstanding that resulted in the ATSB’s failure to collect information from the airplane operator that would have been instrumental in answering questions about the extent of demonstrated deficiencies in flight planning and monitoring.
“Weaknesses in the application of the ATSB analysis framework resulted in those data insufficiencies not being addressed and potential systemic oversight issues not being analyzed,” the TSB said. “Ineffective investigation oversight resulted in issues with data collection and analysis not being identified or resolved in a timely way.”
Although preliminary internal reviews conducted before the release of the final accident report identified a number of issues involving the facts of the accident, many of those issues were not addressed, the TSB review said. The ATSB investigation process did not include a second-level review to ensure that those issues would be addressed, the TSB added.
The TSB noted that the ATSB’s process allotted opportunities for directly involved parties (DIPs) to review accident reports, and in the Norfolk Island case, those DIP reviews were conducted twice — during the initial draft and after revisions had been made.
However, the process did not ensure that DIPs had an opportunity to review the ATSB’s responses to their comments, the TSB said, adding that formal responses to DIPs “increase their understanding of the action taken in response to their submissions and may make them more amenable to accepting the final report.”
High-level ATSB officials reviewed the original accident report only after the first DIP review — some 31 months after the ditching. The TSB — noting that Canadian processes require an upper-level review before a confidential draft is released to outsiders — criticized the timing.
“At this stage in an investigation, it is difficult to address issues of insufficient factual information since perishable information will not be available and the collection of other information could incur substantial delays,” the TSB said. “In any event, there is no robustly documented process after the commission review to ensure that its comments are addressed before the report is finalized. Both of these aspects of the ATSB review process increase the risk that deficiencies in the scope of the investigation and the quality of the report will not be addressed.”
As an example, the TSB cited the ATSB’s handling of a safety issue that was identified during the Norfolk Island investigation — “insufficient guidance being given to flight crews on obtaining timely weather forecasts en route to help them make decisions when weather conditions at destination airports were deteriorating.
“When the safety issue was presented to CASA, it was categorized as ‘critical,’ but in the final report, it was described as ‘minor.’ … The TSB review observed that this shifted the focus of the discussion to the label and away from the issue itself — and the potential for its mitigation.”
The ATSB was unprepared to handle public reaction after the report’s release, even though upper-level officials expected there would be considerable public interest in the document’s conclusions, the TSB said.
“Although senior managers were aware of the possibility that the report would generate some controversy when it was released, communications staff were not consulted and no communications plan was developed to address a controversy if one were to arise,” the TSB said. Those managers were “unprepared for the events that followed” a broadcast report that sharply criticized the investigation, the TSB added.
Comparisons
In addition to its analysis of the Norfolk Island ditching, the TSB review examined final ATSB reports on two other Australian aviation accidents in recent years — the June 15, 2010, crash of a Piper PA-31P-350 in Canley Vale, 6 km (3 nm) northwest of Bankstown, New South Wales, Australia,2 and the Dec. 24, 2011, helicopter winching accident involving an Agusta Westland AW139 in Kangaroo Valley, 16 km (9 nm) west-southwest of Wollongong Airport in New South Wales.3
Both accident investigations were considered similar in scope to the Norfolk Island probe, and in both cases, the TSB described investigations that generally proceeded smoothly.
The Kangaroo Valley accident occurred during a search-and-rescue flight conducted to remove a seriously injured rock climber from a ledge in Budderoo National Park. During the mission, while attached to a winch cable, the climber and a paramedic were pulled off the ledge and struck rocks. The paramedic later died of his injuries, and the climber was evacuated overland to a hospital for treatment. No one else was injured.
The TSB said that the Kangaroo Valley investigation was “a good demonstration of the potential of the analysis tools to structure thinking, test hypotheses, provide a basis for team discussion, assist in identifying the need for additional data and expedite report writing.”
Although investigators exceeded the target timeline and the team leader reviewed reports without also reviewing the accompanying analysis, the process won TSB praise in other key areas.
“The benefits of active engagement with stakeholders are clearly demonstrated through this investigation: Stakeholder expectations for timelines were well managed, additional avenues of data collection were identified and pursued, and timely safety action on all four of the safety issues that had been identified was taken before the report was released,” the TSB said.
“Regular critical reviews during the Kangaroo Valley investigation provided an opportunity for discussion, contributed to revisions of investigation scope and led to data being collected throughout the investigation to support new avenues of investigation.”
Canley Vale
The TSB’s review of the Canley Vale investigation was similarly complimentary.
The crash, which killed both the pilot and the flight nurse — the airplane’s only occupants — occurred as the pilot was attempting to return to Bankstown after an in-flight engine shutdown. The airplane was destroyed by the impact and subsequent fire.
In its final report, the ATSB said that, after the right engine was shut down, the airplane’s speed and descent rate “were not optimised for one-engine inoperative flight” and that it was “unlikely that the left engine was being operated at maximum continuous power as the aircraft descended. As a result, the aircraft descended to a low altitude over a suburban area, and the pilot was then unable to maintain level flight, which led to the collision with terrain.”
The TSB noted that ATSB investigators recognized the need for additional information numerous times during the investigation, and made plans to obtain it, demonstrating “the value of revisiting the data-collection plan as needs and expectations change.”
In addition, the TSB said, “regular contact with stakeholders resulted in timely safety action,” and investigation reviews led to “well-documented decisions and periodic revision of data-collection plans.”
Unlike the Norfolk Island and Kangaroo Valley investigations, this case included a closure briefing — “an opportunity to discuss lessons learned,” the TSB added.
Recommendations
The TSB’s 14 recommendations included praise for the ATSB’s investigation methodology and analysis tools, along with calls to “continue efforts to ensure [their] consistent application and use,” to consider adding steps to ensure that every comment by a reviewer receives a response and to check investigation schedules to determine that realistic timetables are provided to stakeholders.
Other recommendations focused on:
- “Improving investigation methodologies and processes where they were found to have deficiencies;
- “Improving the oversight and governance of investigations”; and,
- “More effectively managing communications challenges.”
Benefits for TSB
From the outset, Canadian investigators saw “potential for mutual learning” in the Australian review project, TSB Chair Kathy Fox said, adding that her agency would re-examine its report to determine whether any ATSB investigation best practices should be incorporated into TSB procedures.
Among the best practices singled out in the TSB review for consideration were the ATSB’s internal preliminary review process in which an investigator not involved in a particular accident investigation is assigned to verify the completeness of an investigation and the form and style of the investigation report, and closure briefings, which discuss internal lessons learned during an investigation.
The TSB also endorsed the “safety summary” sections of ATSB accident reports, which summarize what happened in an accident, the ATSB’s findings, safety actions taken in response and the key safety message.
Notes
- ATSB. Transport Safety Report AO-2009-072, Ditching — Israel Aircraft Industries Westwind 1124A, VH-NGA, 5 Km SW of Norfolk Island Airport, 18 November 2009. 2012.
- ATSB. Transport Safety Report AO-2010-043, Piper PA-31P-350, VH-PGW, 6 Km NW of Bankstown Airport, NSW, 15 June 2010. 2012.
- ATSB. Transport Safety Report AO-2011-166, Helicopter Winching Accident Involving an Agusta Westland AW139 Helicopter, VH-SYZ, 16 Km WSW of Wollongong Airport, NSW, 24 December 2011. 2013.