Even the smallest civil aviation service providers1 today are expected to routinely utilize safety data to assess their current performance and risk indicators, says a team of Flight Safety Foundation researchers. In 2015, they studied safety data collection and processing systems (SDCPS) in the Pan America and Asia and Pacific regions, as defined by the International Civil Aviation Organization (ICAO).
In its first 12 months, the FSF Global Safety Information Project (GSIP) conducted initial research under a cooperative agreement2 with the U.S. Federal Aviation Administration (FAA) into the challenges of creating effective SDCPS worldwide. All stakeholders engaged by the team were collecting increasing volumes of safety data to make better assessments of risk, said Mark Millam, FSF vice president, technical, and GSIP project leader. Yet, the safety experts who participated in GSIP focus groups in 12 cities were candid about impediments, he said in a brief summary of findings and conclusions provided to the FAA.
Among impediments discussed were that some aviation safety professionals interpret data in ways that fit their own objectives; that others construe results of data analysis as only pointing out deficiencies and liabilities to assess blame, rather than opportunities for systemic risk-mitigation advances; and that, in many countries, criminal prosecutions are likely in the event of an aviation accident.
Millam said, “We found many stakeholders begging for a common language for risk management practices so that one stakeholder can easily share its understanding of risk with others.”
As an example of the findings, “Asia Pacific participants expressed concerns that statistical information often is biased by the special interests of the person conducting the analysis,” he said. “This was attributed to both a lack of standardized analysis techniques and to the perspectives and prerogatives of the analyst. Participants additionally reported challenges in identifying root causes of safety issues from the available data.”
In this perspective, key industry-significant factors include a consensus among participants that safety data are a means to solve tomorrow’s problems, but the process of refining data into useful information has not been as consistent or predictable as needed. “Too often, individuals and organizations have come to different conclusions based on information derived from equivalent data processed in significantly different ways,” he said. “There is clearly a need to standardize terminology and approaches to collecting and processing safety data,” including a common taxonomy for risk assessment.
Some participants reported that airline associations and airline alliances are conducting a basic level of safety data/information exchange for specific types of events, and that they sometimes share the broad event-per-exposure rates they are tracking and the responses they have initiated within their safety management system (SMS).
“Our discussion sessions revealed the need for a globally appropriate structure that more completely defines the methods for performing aviation safety data analysis from the very basic levels up through the most complex analysis,” Millam said. “There was considerable interest in ‘measuring ourselves against the rest.’ We will be exploring a means to capture such ongoing assessments via risk assessment and developing a maturity curve.”3
Moreover, focus group participants showed awareness of and strong interest in safety information protection (SIP). Achieving this partly depends on how an organization runs its voluntary safety reporting programs and its commitment to supporting work groups responsible for the hazard-identification process. Specifically, potential data-protection pitfalls while cultivating employer-employee trust include an inconsistent SIP commitment within organizations; inadequate protection across organizations, such as information being divulged beyond the agreed parties when a few believe they have the responsibility to act independently before a consensus is reached; and inadequate protection from harmful news media reports or exposure to punitive judicial actions.
All focus group discussions, and records of those discussions, adhered to a written privacy policy. “We heard that most in the industry knew about and understood the components of an SMS but lacked guidance material on how to manage the SDCPS within an SMS or a state safety program,” Millam said.
Project Elements
In November, the Foundation essentially summarized current global work on SDCPS; how 2016 FSF workshops will develop, introduce and refine GSIP tool kit content about SDCPS and SIP; and how GSIP will support the implementation of related ICAO initiatives.
In the first year, GSIP work activities included the Foundation coordinating with ICAO to name an FSF representative to ICAO’s Regional Aviation Safety Group–Asia and Pacific Regions (RASG-APAC); conducting a comprehensive assessment of the current status of national, regional and global SDCPS; developing a voluntary safety reporting program tool kit for states; beginning tactical work to facilitate the implementation of the legal framework according to guidance added by ICAO in Attachment B, “Legal guidance for the protection of information from safety data collection and processing systems” to Annex 19, Safety Management; and conducting external GSIP-related promotion and communication.
The following examples of key SDCPS and SIP points also were raised in Asia Pacific discussions. Participants questioned how well stakeholders actually can utilize current safety data collection and analysis to understand the inherent risks of aviation from a big-picture perspective, Millam said.
They singled out their uses of flight data monitoring (FDM) as a system that allows for timely operational improvements and procedural corrections to be made. For example, they directly communicate analytical results from hard-landing parameters to all flight crews via landing printouts. Raw, unprocessed flight data also were cited as valuable for making comparisons of normal vs. non-normal events.
Moreover, increased voluntary-reporting participation over time correlated with an increasing level of trust in the rules and processes for providing nonpunitive reports to employers and regulators. One example of a persistent FDM shortcoming was the lack of standardization in how some automated data are collected.
Leaders of some airlines still fail to understand the fundamental benefit of voluntary safety reporting systems, according to some comments in focus group discussions. “Organizations that have failed to see the value may be lacking the just culture of trust and confidence to which other organizations attribute their success,” Millam noted, paraphrasing typical comments, which included skepticism about how well airlines and regulators actually handle, protect and address the risk implications of what they collect and analyze.
A number of Pan American participants emphasized the importance of supporting SDCPS at all levels of a service provider. One consensus point was that collection should not be limited to the frontline staff, but should include the whole organization. A number of states have reported progress toward routine safety data analysis, risk mitigation and the practice of continuous data monitoring, Millam said.
“Voluntary safety reporting programs are being developed in conjunction with the implementation of SMS. These programs have incorporated mobile-device technology into data collection to improve the usability and availability of related systems,” he added.
Nevertheless, among persistent challenges are: organizational cultures resistant to SDCPS; constrained resources; collecting the correct types of data; insufficiency or lack of voluntary safety reports; and inadequate regulation of SDCPS. “The culture of nonpunitive information collection and sharing was reported to be limited in the Pan America region,” he said, paraphrasing expert opinions. “Often, privacy and secrecy are perceived today as signs of corruption. If a data collection process is veiled in secrecy, the program may be viewed as corrupt rather than beneficial to the society’s aviation risk mitigation. Success stories involving safety data collection have not been broadly shared; therefore, the benefits of data collection are still questioned amid the longstanding cultural beliefs.”
Others from Pan America described fear among some operators that their regulator will issue sanctions based on the results of sharing voluntary safety reporting information. Also, flight operations and maintenance inspectors often approach their jobs and inspections from a perspective of policing — rather than working with the operator to improve safety, he said.
SIP varies from state to state, and changes based on whether a given group is from the private sector or government/military. “Some states’ data protection laws are well established, while others’ are newly implemented and have yet to be fully tested. Therefore, the effectiveness of the laws is unclear,” Millam said.
SIP-Related Findings
GSIP strategic planning and background research has yielded state-level facts such as whether a mandatory and/or voluntary reporting system is in place; references to existing legislation, regulations and policies governing SIP; and other SIP notes related to the procedure of protection for voluntary reports (analysis and sharing of safety information) and recent news on SIP in the state, if any. Also covered were the civil aviation system of each state, including how civil aviation activities currently are regulated and how the relevant competent authorities cooperate on SIP under existing legislation, laws, regulations, policies and advance arrangements.
To the extent that information was available, GSIP findings have covered legal protection in terms of confidentiality of safety reports, applicable evidentiary laws, protection against the disclosure of safety data and information, and provisions on the assurance against prosecution or punishment.
Next Steps
In the next phase of GSIP, the Foundation will design a tool kit with input from previous participants and new subject matter experts, and use online collaboration and live workshops to vet tool kit content. Millam said, “For example, preliminary plans call for the tool kit to detail the maturity curve process that enables stakeholders to compare their efforts with the risk-management processes and safety information protection implemented by peers in other ICAO member states.”
The Foundation expects the tool kit to provide detailed examples of best practices and insights into advanced types of risk analysis, data sharing and SIP already being used successfully. An FSF internal summary of work to date noted, “Tool kit content about the ICAO legal framework for SIP, as developed by its SIP Task Force, is expected to provide specific assistance to states interested in advocating for legislative and regulatory changes that will provide assurances that critical safety information will remain intact for the foreseeable future.”
Summarizing GSIP’s expected path4 through 2016, Millam added, “As we embark on building tool kits, our work will take time to organize with the help of key experts, but it is a necessary step in enhancing safety by learning from data before accidents happen.”
Notes
- These include approved training organizations, operators of airplanes or helicopters authorized to conduct international commercial air transport, approved maintenance organizations, organizations responsible for the type design or manufacture of aircraft, air traffic service providers and operators of certified airports.
- The cooperative agreement is the form of legal contract under which GSIP activities are funded by the FAA.
- Maturity curve, in this context, refers to a type of graphical scale for comparing relative SDCPS complexity and capability using defined process elements.
- For the latest details, see <fsfgsip.org>.
