Partners and Programs:
Partners and Programs:

FLIGHT SAFETY FOUNDATION HEADQUARTERS

701 N. Fairfax Street, Suite 250,
Alexandria, Virginia 22314

Phone: +1 703 739 6700 Fax: +1 703 739 6708

GSIP Related Articles, Threat Analysis

Sharing the Wealth

Vulnerability-discovery methods of ASIAS gain acceptance across national borders.

by Wayne Rosenkrans | March 29, 2013

Creators of the Aviation Safety Information Analysis and Sharing (ASIAS) program in the United States imagined years ago that their methods could be a good fit for other countries and their airlines and possibly non-airline interests within aviation. Currently, this government-industry partnership funded by the U.S. Federal Aviation Administration (FAA) has about a year of experience working closely with a number of similar international partnerships, said Jay Pardee, the FAA’s chief scientific and technical advisor for vulnerability discovery and safety measurement programs, in a recent briefing for AeroSafety World.

According to FAA, ASIAS leaders also expect to fufill their intention to “increase the numbers and types of participants following a phased expansion plan to include other parts of the aviation community. ASIAS will include domestic corporate general aviation, military, helicopter, manufacturers and other government agencies.” An example of effect on another government entity is the relationship forged a few months ago with the National Transportation Safety Board (NTSB; see “Agreement Launches ASIAS–NTSB Working Groups”).

Agreement Launches ASIAS–NTSB Working Groups

The U.S. National Transportation Safety Board (NTSB) has gained access, on a case-by-case basis, to a vast store of summarized safety data gathered, analyzed and protected within the Aviation Safety Information Analysis and Sharing (ASIAS) program to prevent airline accidents (Table 1, p. 44).

When a formal NTSB request is approved under the November 2012 memorandum of understanding, the board reciprocates by granting ASIAS access to archived digital flight data recorder (DFDR) data specifically related to the request. Regardless of whether an accident occurs, ASIAS is working with the NTSB to acquire archived DFDR data that, when added to the ASIAS databases, might reveal accident precursors or indications of systemic risks. Various restrictions preserve the U.S. airline industry’s voluntary — and now indispensable — flows of safety information from routine flight operations, according to the NTSB and the U.S. Federal Aviation Administration (FAA).

Collaboration under this highly structured and controlled framework stands to enhance all parties’ predictive methods (ASW, 3/13; ASW, 11/11), said Paul Morell, a captain and vice president, safety, security and environmental programs, US Airways, and industry co-chair, ASIAS Executive Board (AEB). From his perspective, most noteworthy is that the memorandum’s provisions only apply in an accident involving a U.S. air carrier in the United States.

“In essence, we’re expanding the database for ASIAS in order to do our research,” Morell said. The ultimate advantage to ASIAS members will be gaining new insights from the NTSB relationship and carrying over data that help the Commercial Aviation Safety Team to develop effective risk mitigations, he added.

Any accident that meets these criteria provides “NTSB an opportunity to talk to our IAT [ASIAS initial analysis team tri-chairs] and AEB co-chairs, and to see if there could be something, some kind of information we may have where we can help them in their investigation,” Morell said. Ideally, knowledge will expand beyond an isolated case being investigated by NTSB, he said.

NTSB-ASIAS interaction for a given event begins with these investigators. “First of all, there’s a protocol within the NTSB to determine whether they will ask ASIAS to engage,” Morell said. “Once they make a determination within the NTSB, then there is the informal query, NTSB talking to the IAT tri-chairs. We either determine ‘we can probably help you’ or ‘we really don’t have what you’re looking for.’ Once we get the formal request, then we engage and we meet at MITRE Corp. [the not-for-profit, FAA-funded research center that has stewardship of ASIAS data]. We have the NTSB representatives and we have the IAT representatives create a working group.”

The working group uses the formal request as its scope of work but with a practical degree of flexibility and adjustments that are made possible — but not guaranteed — by another mechanism. “If they say ‘we need to take this little turn over here because we discovered something related during the investigation,’ and if it’s within the line and the framework of the initial permission that was given for the setting up of the work group, then they can continue doing that,” he said. “If it’s outside of that scope, then the ASIAS Executive Board will make a determination whether to allow them to do that.”

The memorandum bars any of the parties from using FOQA data, aviation safety action program reports, air traffic safety action program reports or non-publicly available data to measure an individual data contributor’s performance or safety. ASIAS protocols already had limited FAA analysts’ access to aggregate and/or de-identified information for purposes outside of ASIAS.

“One of the benefits I see in looking at the NTSB DFDR data archives is that we can use those signatures, those digital data patterns, bring them into our vulnerability discovery activity within ASIAS, and then digitally look across all of the digital databases to see if we see any indication of that same pattern,” said Jay Pardee, the FAA’s chief scientific and technical advisor for vulnerability discovery and safety measurement programs. “Not necessarily [studying] the accident itself, but we’re interested in the precursors that would be in that DFDR trace prior to an event.”

“The absolute cornerstone of ASIAS is that nothing leaves the working group until the NTSB and the IAT make a determination that it should become part of a public record or part of their investigation,” Morell said. “At that point in time, all that is brought to the AEB, and the Executive Board will make a determination whether or not that can be released. If not, then the NTSB can’t use it in their report but they still have an idea of where they should go or what they should do.”

For example, if the NTSB investigates a commercial jet runway overrun accident involving excessive time elapsed between touchdown and the flight crew’s spoiler deployment, the NTSB will not be permitted to use any aggregate ASIAS information that could be used to compare the accident crew’s performance with the industry norm.

However, if a commercial jet landed short of a runway after experiencing fuel system icing during approach to an airport — and ASIAS-member airlines’ FOQA and engine data were accessed to support this investigation — these circumstances likely would be favorable for concurrence by the AEB in NTSB’s desire to publish comparisons of the accident scenario with multi-airline experience.

“There’s a perfect example where you’re using information from all these sources but you’re not comparing an airline or a crew, you’re looking at a generic view of how a system operates,” Morell said.

— WR

As when it launched in October 2007, ASIAS continues to focus on known-risk monitoring, vulnerability discovery and directed studies typically prompted by findings of ongoing safety data analyses, the needs of the Commercial Aviation Safety Team (CAST) and concerns raised by the U.S. airline industry during semi-annual Aviation Safety InfoShare meetings (ASW, 5/08). The system’s techniques, data sources and algorithms have appealed to several countries, but patterns and structures recently created for the Latin American and Caribbean Region have yielded an especially sound model, Pardee said.

“We’ve forged a relationship with some additional regions and countries around the world,” he said. “The Regional Aviation Safety Group–Pan America [RASG-PA] uses the same principles that we use in ASIAS and CAST; it is an industry-government partnership [ASW, 2/13]. The ASIAS Executive Board has agreed to share our U.S. airline member experience flying into some 22 airport locations” with RASG-PA’s industry-government teams representing those airports. A memorandum of understanding effective in January 2012 defines the ASIAS–RASG-PA working relationship.

“The information is in an aggregated, protected, de-identified state, which is the way we use it under our governance within ASIAS,” Pardee said. “They’ve adopted at this point in time 11 CAST safety enhancements. Many of them were built based on knowledge from ASIAS, which we share to assist RASG-PA in implementing solutions for problems we have knowledge about, that we have experienced in our own country. They’ve agreed they will adopt 33 starting with those elements that are the most logical for their safety issues.”

The focus during most of the first year of the relationship was providing quarterly safety information reflecting ASIAS member airlines’ experience operating into the 22 RASG-PA airports, as an indication of the effectiveness of their risk-reduction solutions. Members of a U.S. government-industry issue analysis team provide technical data that identify the safety issues relevant to unstabilized approaches, terrain awareness and warning system (TAWS) alerts, traffic-alert and collision avoidance system (TCAS) resolution advisories (RAs), runway excursions and other potential threat indicators.

“We help measure the effectiveness of actions in their region’s airports by looking at our own ASIAS information and — in a protected fashion … a de-identified fashion — sharing that with the [RASG-PA] industry-government cochairs,” Pardee said. “The advantage is, from the ASIAS airline member perspective, we can achieve improvements in the safety of airports that our airline members fly into within Pan-American countries. It has been a very successful experience for ASIAS and the RASG-PA organization — a model we do intend to use around the world. We periodically review the data with them, look at the progress they’ve made, and it’s a relationship that continues to prosper. ”

The ASIAS–RASG-PA collaboration has been able to delve deeply into threat detection and mitigation monitoring partly because of mutual trust, he said. “We get down to looking discretely at the details of those types of precursors, and we monitor the frequency and location jointly with RASG-PA helping them to take corrective action,” Pardee said. “Airlines that are domiciled in that region benefit from the same improvements as U.S. airlines in safety, CFIT [controlled flight into terrain] reduction, improvement in unstabilized approaches, reduction in TAWS alerts, reduction in TCAS RAs, so we are using the data effectively.”

NextGen Safety Assurance

One of the most recent ASIAS directed studies — focused on operations using area navigation (RNAV) off the ground (ASW, 3/12) — was completed, and study-based safety enhancements currently are under development by CAST, said Michael Basehore, FAA’s ASIAS program manager.1 The latest directed study — focused on STAR (standard terminal arrival route) RNAV operations — is looking at risk factors in the context of a wider NextGen research program that began in 2010, Basehore said. Called Optimization of Airspace and Procedures in the Metroplex (OAPM), the program has been studying the benefits of changes to airspace and procedures through local teams in 21 NextGen-defined metroplexes.2

“[ASIAS works] with these OAPM teams as they look at the different metroplex areas, and shares with them the information that we’ve gained from a directed study, known-risk monitoring or benchmarks that we’ve already identified for their metroplex,” Basehore said. Notably, he added, this input — unlike retrospective safety analysis — occurs before airspace, route or procedure redesign is even initiated. ASIAS has collaborated to date at three metroplexes, raising awareness of issues such as non-safety-critical TCAS RAs when general aviation and commercial air transport operators operate in adjacent airspace (ASW, 8/09, p. 34).

“ASIAS, in a protected fashion, contributes what the known tactical threats are in those areas, whether they are TAWS warnings, TCAS alerts or unstabilized approaches,” Pardee said. “We provide that safety information to metroplex teams along with the tools that ASIAS developed to detect them in the first place.” Two software modules identify TAWS-warning hotspots and TCAS RA hotspots, patterns detected through analysis of radar tracks.

The modules are plug-ins to the NextGen airspace redesign software called Terminal Area Route Generation Evaluation and Traffic Simulation (TARGETS). “As a new airspace or route or procedure is developed, it’s done with knowledge of where the current tactical safety concerns from ASIAS are using this airspace-design tool, which has the detection algorithms built into it” for preemptive risk reduction, he said.

A similar activity recently begun by ASIAS to strategically “design risks out of the system” throughout NextGen implementation has been detection of tactical safety concerns during local adjustments to new arrival procedures based on performance-based navigation (PBN), Pardee said. “As new RNAV arrivals, as an example, are being designed, we take advantage of the opportunity to address our TAWS issues, and lead airline members are part of that activity,” he said. Another activity has been the preparation of safety-assurance metrics for NextGen, defining evidence of the required level of safety.

“Over the past eight months we’ve reached out to the different NextGen portfolio managers,” Basehore said. The message for them covered ASIAS capabilities relevant to NextGen planning, risk baselines, and conducting the post-implementation determination of successes/failures and associated risks.

Throughout ASIAS, new safety metrics — for example, geographic distribution of anomalies and adverse trends such as TCAS RAs with their contributing factors — also are undergoing refinements to ensure useful information can be produced as NextGen precision evolves. “We could then get a better handle on understanding locations, flows and maybe more importantly, to categorize or classify the severity of the various penetrations of these safety barriers. That’s a strategy that I see us delving into further this year,” Pardee said.

Among raw materials that ASIAS analysts recently have begun to fuse with other sources are recordings of automatic dependent surveillance–broadcast system (ADS-B) messages, and voice communication between pilots and air traffic controllers, FAA said in its 2012 NextGen implementation report.3 “We’re starting to look into the possibilities of digital voice-track data,” Basehore said. “In some of the studies not knowing the conversation that was occurring between the controller and the pilot has left us in a lurch. For example, we’d see in a departure an aircraft leaving the procedure, but without analysts knowing whether there was a conversation where the controller said ‘yes, you can cut the corner short’ or the pilot said that he was turning for avoidance conditions. We’re in the research phase of actually acquiring voice-track data to meld it with data for particular potential safety issues that we have seen.”

Fully understanding an altitude deviation during an arrival, as another example, could require this capability. “Our research right now is about getting that voice data and being able to ‘pin it’ to that particular radar track to get the full realm of what’s going on,” he said.

Although not new to ASIAS, data mining of narrative texts — including auto-classification of reports by computer algorithms — has become more sophisticated for sources such as aviation safety action program (ASAP) reports. “We have seven ASAP trends that we monitor on a regular basis, and probably five or six more under development so that we can mine the text data just the way we mine the numerical digital data” except for calculating trends not rates with ASAP reports, Basehore said.

The NextGen implementation report cited ongoing work by ASIAS, including “helping the FAA and stakeholders with better characterization and understanding of missed approaches, runway overruns, rejected takeoffs, autobraking and energy states on final approach. This nuanced understanding is expected to aid in accident prevention.” The report pointed to ASIAS initiatives to develop a new method to query multiple databases with one search directive; add air traffic control (ATC) facility-performance data from FAA’s Air Traffic Organization to analyze safety effects of unplanned service interruptions; develop data standards and integration capabilities to add digital flight data from voluntary sources in general aviation, especially de-identified aggregate data from corporate flight operational quality assurance programs (C-FOQA); improve the query and visualization software on secure Web portals used by ASIAS members; and revise data standards for FOQA data sources and sources of voluntarily submitted text reports.

Winning Over Airlines

Venues long used by U.S. airline directors of safety to confidentially share lessons learned, methods, anecdotes and trends among peers have not been superseded. But one airline’s perspective illustrates the relative influence of ASIAS, said Paul Morell, a captain and vice president, safety, security and environmental programs, US Airways, and industry co-chair, ASIAS Executive Board.

“At US Airways, our SMS [safety management system] deals with FOQA, ASAP, a lot of the things that are going on with ASIAS,” Morell said. “But we’re very limited in the scope of what we’re looking at.” For him, the key advantage of ASIAS has been the company’s ability to tap ASIAS databases “to look at aggregated data or different airports or different types of data and to also compare and use the Web portal dashboards” to analyze issues such as unstabilized approaches.

He explains, “I might be thinking we’re doing really well, but I can compare US Airways against the aggregate. … I can see that maybe I have a problem at one airport, but am I the only one that has that problem?”

For Morell, the second advantage is the ability to contribute effectively to the safety of the entire airline industry, without diminishing the value of sharing experiences with safety committees of airline associations, academia and industry initiatives. “ASIAS analysts can go in there, and look, and see where I might not even think I have a problem personally at our airline but by the small amount that I’m contributing, and different airlines are contributing now, I’m enabling a ‘larger SMS’ — because that’s what ASIAS is.”

All told, this cycle — threat identification by ASIAS and InfoShare, then risk mitigation and systemwide solutions through CAST safety enhancements, and finally ASIAS measurement of risk mitigation effectiveness — has been a widely welcomed advancement, he said. 

Notes

  1. FAA. NextGen Implementation Plan. March 2012. In 2012, ASIAS was involved in a directed study of risks while FAA validated the safety and capacity benefits of implementing RNAV off the ground in three of 21 NextGen metroplexes, designated as Houston, Memphis and North Texas.
  2. FAA. The report says, “OAPM is a systematic and expedited approach to implementing PBN procedures and associated airspace changes in major metropolitan areas. Expected improvements from OAPM include efficient descents, diverging departure paths and decoupling of operations among airports within the metroplex airspace.”
  3. FAA. The report says, “The aim of [OAPM] is to have study groups identify near-term PBN improvements coupled with airspace sector adjustments that can be completed in major metropolitan areas within three years.”

Print: