The Aviation Safety Information Analysis and Sharing (ASIAS) program of the U.S. Federal Aviation Administration (FAA) credits strong airline support and a joint industry-government approach for today’s wide acceptance of its strategy, methods and products. Four years after its launch, and contrary to early concerns, ASIAS analysts have not been hampered by the agreed boundaries around use of airline data to identify safety solutions. Rather, the most pressing challenges now include focusing resources based on sound safety-risk assessment and delivering the desired vulnerability-discovery capability, said Jay Pardee, FAA chief scientific and technical advisor for vulnerability discovery and safety measurement programs, and Michael Basehore, ASIAS program manager.
The participation and funding levels are “a testament to the value that both our airline members of the ASIAS community and the FAA attach to our activities,” Pardee said. As of November, the program had 40 U.S. airlines contributing experience from flight operational quality assurance (FOQA) programs, aviation safety action programs (ASAPs) or both. Each airline has signed a memorandum of understanding (MOU) with the Center for Advanced Aviation System Development at the MITRE Corp., a federally funded research and development center, to provide ASIAS analysts network access to de-identified FOQA and ASAP data — and to be among the first to receive analytical reports and industry safety benchmarks derived from aggregation and/or fusion of airline data with more than two dozen non-airline datasets.
“There has been a significant increase, to at least double the number of members in ASIAS since August 2009, further improving the statistical significance of airline datasets and ensuring even more robust coverage of certain locations and aircraft types,” Pardee said. “Our basic method of operation is working as effectively as ever. All parties today have a much higher degree of confidence in program governance1 and preservation of confidentiality and less concern about how ASIAS work is being undertaken.”
The amount of proprietary airline data on the network reached a level suitable for statistically significant analyses of system safety issues during the first two years of ASIAS. Continued growth of airline participation has been beneficial, but some missing pieces also have been recognized. “In some instances, we are oversubscribed in certain aircraft types — we already have a lot of FOQA data and ASAP reports for them,” Basehore said. “Now, we are focusing more on aircraft types for which we don’t have as large a database, and particular geographic locations for which we lack data. But we are still encouraging any airline that wants to participate to join, and we will actively work with them.”
At a time of strained government resources, data analysis on this scale has to be conducted using a risk-based strategy focused on strictly limited datasets, Basehore said. As of April 2011, the ASIAS network could analyze FOQA data from 7.7 million flight operations, 83,000 ASAP reports and 30,000 air traffic safety action reports.
“If we spent our time trying to look at every single data point, we would quickly exhaust our funding,” he said. “Some issues found obviously are riskier than others. We make sure that we take that into account.”
The first recipient of analyses outside of ASIAS — and the entity responsible for developing voluntary, system-level safety enhancements — is the U.S. Commercial Aviation Safety Team (CAST). In October, the FAA received public comments about its intention to collect safety-related data regarding the voluntary implementation of CAST safety enhancements by U.S. air carriers.
Each safety enhancement approved by CAST represents a commitment of sufficient resources by the FAA and the U.S. airline industry, Pardee and Basehore said. For example, aircraft and avionics manufacturers commit to the associated design functionality improvements, and airlines commit to upgrade their aircraft, change flight crew training and take other related actions.
Most of the latest CAST safety enhancements — out of a total of seven derived from ASIAS work — mitigate non-safety-critical traffic-alert and collision avoidance system (TCAS) resolution advisories (RAs) using “local deconfliction of traffic to reduce the frequency of TCAS RAs and opportunities for short-term adjustment to the software algorithms in the TCAS unit itself with ground-based radar inputs changing the sensitivity of TCAS hardware,” Pardee said. “We’re also looking for opportunities to design the airspace of the future based on the TCAS RA information acquired from meeting with airlines and the work done through ASIAS.”
Other safety enhancements described in previous articles (ASW, 5/08, p. 25, and 8/09, p. 32) focused on non-safety-critical alerts from terrain awareness and warning systems (TAWS). An example of attention to a relatively old safety issue is continued ASIAS monitoring of routine operations for evidence of controlled flight into terrain (CFIT) risks. Other issues still monitored include the risks — and the effectiveness of CAST safety enhancements — adopted years ago to mitigate approach and landing accidents, runway safety threats, mid-air collisions, loss of control in flight, icing, cargo operations threats and maintenance threats.
ASIAS monitoring of the older CAST safety enhancements has disclosed successes and shortcomings. “There are elements we can identify that warrant further improvement,” Pardee said. The recent monitoring of unstabilized approaches, for example, led ASIAS analysts to look beyond the specific airports, runway ends and arrival procedures studied originally. “One CAST safety enhancement encourages pilots conducting an unstabilized approach to execute a go-around, but that is an example of where we need action to further improve how effectively that solution is working,” he said.
Vulnerability Discovery
From the outset, the FAA expected vulnerability discovery — the recognition of new risks, threats and system-level precursors not revealed by forensic investigations — to become a core competence of ASIAS. The intention was to ensure constant vigilance for anomalies/atypicalities and to complement the formal directed studies, known risk monitoring, safety enhancement assessment and benchmarking of safety in airline operations.
Developing a true capability for vulnerability discovery has particular importance for the Next Generation Air Transportation System (NextGen), the FAA’s transformation of U.S. airspace that, among other things, will replace radar surveillance with satellite-based surveillance of air traffic. The primary role of ASIAS in NextGen implementation is to provide safety assurance information, as defined by the FAA’s internal safety management system, Pardee said.
“Vulnerability-discovery capability is a work in progress, still maturing … we are still learning, developing and perfecting our skills,” he said. “Our latest methodology has been to use lessons learned from the forensic history to identify undesired aircraft states.
“The forensic history tells us that if an aircraft enters one of these undesired aircraft states, the outcomes usually constitute a safety threat.
Accepting that fact — from the perspective of not knowing the causes why an aircraft could enter one of these states — we have begun to exercise our capability to look, for example, at what might be significant FOQA exceedances in roll or bank. We begin by looking through many ASIAS databases just for the existence of undesired aircraft states. We then let the data take us where we should be looking rather than presume we understand all the potential ways that an undesired aircraft state could occur. We look at indications from the data — atypicalities and anomalies, unexpected changes in rate of exceedances and try to compare those.”
Current directed studies by ASIAS of area navigation (RNAV) off the ground reflect the early-warning role of ASIAS. “NextGen is based on using RNAV procedures as one of the larger components, so by looking at these procedures as they are introduced, we are out in front in implementing NextGen,” Basehore said. “So ASIAS is now looking at RNAV departure and arrival procedures at certain locations, making sure that we fully understand the changes when the FAA puts those procedures in place — how they affect both the operators and the FAA Air Traffic Organization. If we can’t get RNAV right, we are not going to get NextGen right.”
As methods evolve, new databases are added to the ASIAS network and lessons emerge from analytical experiences, Pardee and Basehore expect to continue shifting the FAA’s emphasis from forensic to prognostic aviation safety improvements. The completed directed study of unstabilized approaches was a recent example, and results of directed studies of airport construction threats will be presented to CAST when completed.
“For ongoing study of unstabilized approaches, we now can locate — with the aggregate, de-identified FOQA data — particular airports and procedures that possibly result in a larger number of unstabilized approaches than others,” Basehore said. “We also have been able to start looking at weather related to a particular airport and FAA air traffic surveillance data, so we no longer have to rely strictly on the FOQA data from the airlines. For some of the metrics … we are now able to merge data such as what the weather was and what approaches were used on a particular day, so we get a much better feel for what happened than before we were able to fuse and merge all the data sources.”
The ASIAS program in the past two years has tapped some federal government datasets for the first time. “Although we still work with the protected FOQA data, we can bring into analytical proximity many more databases — such as all of the FAA radar surveillance data that were not available early in the ASIAS program,” Basehore said. “These enable more detailed work and a much more robust understanding. From my perspective, with these new sources and advances in analyzing numerical data and narrative data, we have not encountered any obstacles in carrying out our safety activities while still abiding by the ASIAS principles of governance.”
“We have not undertaken any work that we could not complete because of the MOUs regarding de-identified aggregate data,” Pardee added.
Infoshare Prominence
The twice-a-year FAA-industry meeting called Aviation Safety Infoshare has become the primary means of communication about ASIAS activities. The next revision to the compact discs distributed to the industry by CAST will reflect the voluntary safety enhancements adopted based on ASIAS studies of TAWS and TCAS.
“Infoshare meetings are now regularly connected to ASIAS as a source of information and potential concerns for us,” Pardee said. “These meetings have become an opportunity for us to engage with many of our ASIAS-member airlines and non-ASIAS members, and for them to share safety issues, concerns and experiences with the FAA and among themselves. … In many cases, Infoshare discussions become an affirmation of what we think we see in ASIAS data and what we have acted on in the past through CAST. In other cases, there may be nuances of a prior issue raised or the beginnings of an operator experiencing something that we haven’t focused on before.”
Basehore noted that 2011 Infoshare discussions demonstrated an improved integration of voluntary safety programs within individual airlines. Safety teams sent by airlines to these meetings typically had both FOQA representatives and ASAP representatives prepared to jointly present perspectives of the same safety issues, he said. Some U.S. and non-U.S. airlines pointed out ASIAS-like internal techniques of fusing FOQA and ASAP databases (or international equivalents) for company-level analysis.
U.S. airlines that have not signed an MOU still have access to most information generated by ASIAS, and from the FAA’s perspective, they have not been impeded in risk-reduction activities. “Airlines that participate directly in ASIAS certainly are involved in the directed studies and receive early safety information,” Pardee said. “But a tremendous number of operators — members of ASIAS and non-members — attend Infoshare meetings, listen to our description of the ASIAS products and results and observe the safety information sharing among operators. Safety enhancements and solutions are shared by ASIAS throughout the airline community and all of their associations, so there are multiple paths to receiving ASIAS-developed, CAST-executed products.”
At the international level, data-sharing and analytical processes are maturing under 2011 agreements among the European Union, FAA, International Air Transport Association and International Civil Aviation Organization (ICAO), he added. “We are sharing common taxonomies between regulatory authorities and ICAO, for example,” Pardee said. “We have defined things like the classification of incident precursors as well as the criteria … for applying digital measurements to events like unstabilized approaches, and we have shared definitions more universally. Non-U.S. entities similar to ASIAS are beginning to use the same taxonomies. The FAA partners in the sharing of safety information and measurements of the effectiveness of safety solutions from CAST and ASIAS with other CAST-like safety organizations, such as emerging regional aviation safety groups sponsored by ICAO.”
Note
- The basic principles of governance are using data solely for the advancement of safety, using de-identified airline data, non-punitive reporting and approval of analyses by the FAA-industry ASIAS Executive Board.
