The International Civil Aviation Organization and Flight Safety Foundation are among numerous organizations in civil aviation that extol the importance of efficient and effective safety data processing and collection systems (SDCPS) in improving civil aviation safety performance. But the collection and analysis of safety data also can be used to identify and mitigate risk in military aviation.
Recently, the U.S. Government Accountability Office (GAO), which investigates for the U.S. Congress how the federal government spends taxpayer dollars, said it found gaps in the Department of Defense’s (DOD’s) approach to collecting, reporting and analyzing military aviation mishap1 data, and that DOD needs to improve its approach to managing risks. Also, GAO said it found a lack of consensus between the Army, Navy and Air Force safety centers2 and the Office of the Secretary of Defense (OSD) on the reporting of data to OSD on human factors that may have caused the mishap, and that DOD does not consistently collect and analyze relevant training data from all mishap investigations. “Recent studies have suggested that training shortfalls are a potential indicator of trends in aviation mishaps,” GAO said.
The analysis of DOD’s data collection and processing efforts was done as part of a wider, ongoing study of military training for rotary-wing aviation stipulated in the National Defense Authorization Act for fiscal year 2018. According to an Aug. 15 report to Defense Secretary James Mattis, GAO, as part of its review, is evaluating the relationship between training completed on rotary-wing aircraft and the number of aviation mishaps that have occurred. The final report stemming from that review is expected in early 2019, GAO said, but given what it knows about DOD’s efforts to address aviation mishap–related policies and oversight issues, GAO opted to share some of the issues it identified during its ongoing review.
To evaluate the extent to which DOD has a comprehensive approach for the collection, reporting and analysis of mishap data to inform risk-management decisions, GAO investigators reviewed DOD Instruction 6055.07, Mishap Notification, Investigation, Reporting, and Record Keeping, which establishes the roles and responsibilities of the offices involved in the collection and reporting of aviation mishap data, as well as agreements between the OSD and the safety centers on sharing data. According to the GAO report, DOD Instruction 6055.07 says that standardizing data across all of the DOD components “provides a common language for all mishaps, and that the use of standard data allows for accurate mishap trending, efficient hazard analysis, and more effective sharing of lessons learned.”
GAO said it compared the mishap data collected by the safety centers and reported to OSD as of May 2018 against the mishap data elements in the current agreement, reviewed documentations and interviewed officials from OSD and the safety centers.
Each of the military departments, largely through the safety centers, is responsible for the notification, investigation and reporting of aviation mishaps. OSD is responsible for collecting and analyzing mishap data from the safety centers to provide DOD leaders with information on risks, recommendations on mitigation strategies and distributing lessons learned within and outside DOD.
Among GAO’s findings is that the safety centers do not collected standardized mishap data despite various policies and agreements with OSD to collect uniform data for specific mishap data elements. From 2000 through 2017, OSD executed three separate agreements with the safety centers related to collecting and sharing standardized data with OSD. The most recent agreement, an August 2017 memorandum of understanding (MOU), identified 35 aviation-related mishap data elements that should be collected and reported by the safety centers in a standardized way. But in May, GAO found that the safety centers collected 11 to 17 of the 35 elements, depending on the service. The safety centers either did not collect the other elements or did not collect them in a standardized format, GAO said. For example, the Army Combat Readiness Center and the Naval Safety Center do not collect data for the “area of responsibility” element that specifies under which combatant command the mishap occurred.
OSD officials told GAO that OSD must perform time-consuming manipulation and interpretation of certain data elements received from the safety centers because the centers collect nonstandard data, which introduces the risk of errors in analysis. For example, for the data element “aviation mishap type,” the safety centers are to choose one of 20 standard values, such as “midair collision” or “wildlife strike,” to characterize the mishap. However, the Army collects these data using its own process, whereby it selects up to three of 102 values to describe the aviation mishap type.
The GAO report also said that interviews with officials at OSD and within the service branches indicated that there is not a consensus between the safety centers and OSD regarding OSD’s role in conducting causal analysis of mishaps. As a result, the safety centers do not report all data elements to OSD, including the causal factors related to mishaps, despite their agreement in the August 2017 MOU. As of May, the Army and Navy safety centers were not reporting four of the 35 agreed data elements, and the Air Force Safety Center was not reporting seven of the 35 elements. None of the safety centers was reporting information on human factors that contributed to the mishaps, which according to DOD represents the leading cause of DOD mishaps.
DOD also does not consistently collect relevant training data to analyze trends in mishaps. For example, an official from the Air Force Safety Center told GAO that a pilot’s recent flying hours may not be captured if the investigator does not believe that the information is relevant to the mishap.
In its report, DOD made three recommendations for executive action. The first is that Mattis ensures that the undersecretaries of defense for personnel and readiness and for acquisition and sustainment, in coordination with the secretaries of the Army, Navy and Air Force, take interim steps to help ensure that standardized aviation mishap data elements are collected by the safety centers; that department-wide and service instructions and policies clarify OSD’s access to human factors information and its responsibility for conducting analysis; and that Mattis — working with the DOD chief management office and various undersecretaries, and the secretaries of the Army, Navy and Air Force— identify relevant training-related data to collect as part of any update of the aviation mishap data elements and incorporate these data into future analyses.
GAO said a draft of its report was shared with DOD for review and comment, and that DOD concurred with all three recommendations and stated ongoing and planned actions it is taking to address the recommendations.
Notes
- An aviation mishap involves a DOD aircraft or flying operations and is defined as an unplanned event or series of events that results in damage to DOD property; occupational illness to DOD personnel; injury to on- or off-duty DOD military personnel; injury to on-duty DOD civilian personnel; or damage to public or private property, or injury or illness to non-DOD personnel, caused by DOD activities.
- The branches of the U.S. military that fall under the auspices of the DOD are the Army, Navy, Air Force, Marine Corps and Coast Guard. The fifth branch, the U.S. Coast Guard, is under the operational control of the Department of Homeland Security during team, but can be placed under the control of the Navy in war time or by presidential order. The military safety centers referenced in the GAO report are the U.S. Army Combat Readiness Center, U.S. Air Force Safety Center, and the U.S. Naval Safety Center, which also manages the Marine Corps portion of the Naval Aviation Safety Program.
Featured image: © Riko Best | Adobe Stock