Future error-tolerant system improvements to airline operations will require a fresh appraisal of voluntary safety programs to ensure they match advances in safety culture and technology, says Timothy Logan, director of safety, Southwest Airlines. He urges the industry and government to come to grips with legacy philosophical errors that today affect how well these programs support safety management systems (SMSs). He presented his paper and answered questions in August during the ISASI 2012 seminar in Baltimore organized by the International Society of Air Safety Investigators (ISASI).
“It’s really important to talk about the state of these programs,” Logan said. “[There’ve] been a lot of things underlying these programs that need to be fixed, [that] need to be rectified for us to have a really good SMS. … For those of us in the safety offices of large operations, our focus is no longer on reacting to hull losses or even serious incidents. Our focus has moved toward preventing incidents through the identification of hazards and threats for which we previously had little to no information.”
SMS rests on four pillars: safety policy, safety risk management, safety assurance and safety promotion. He called safety risk management and safety assurance “the engines of SMS” in everyday airline operations because they drive decision making. Yet today’s voluntary safety programs are not structured to support the system safety process of SMS, he said.
In the Southwest Airlines SMS, these programs are a subset of all information sources used in system safety analysis and decisions. Among other sources are irregularity-reporting programs, occupational injury reports, internal evaluation program audits, maintenance reliability programs, aircraft damage reports and internal investigation reports.
The voluntary safety programs he discussed were aviation safety action programs (ASAPs) for specific employee groups; flight operational quality assurance (FOQA, also called flight data monitoring of routine operations); voluntary disclosure reporting programs to the U.S. Federal Aviation Administration (FAA) by the airline and its maintenance, repair and overhaul facilities; line operations safety audits (LOSAs); and FAA-sanctioned internal evaluation programs. Other airlines also rely on their FAA-airline advanced qualification programs (AQPs), Logan said.
“We have lots of data, and the problem is [that] it comes in waves, very unorganized, and we are sitting at the bottom [of the figurative waterfall] with the pail trying to figure out what are those issues to go look at,” Logan said. “What’s interesting is that, in a lot of cases, those [sources of safety data] aren’t shared outside of the airline. But … we have to merge that data with our ongoing data … from our maintenance programs through reliability [studies]. We have damage reports, we have injury reports.”
Injury reports, for example, don’t include a flight number or a date. “We don’t even necessarily know how to relate [an injury report] to the flight or a facility,” he said. “It is so frustrating sometimes to have a report that has good information in it, but we can’t use it to be able to [relate] it back to a flight or a maintenance facility.”
Origins of Problems
From Logan’s perspective, airlines are inhibited by the independence of these programs from each other, the conflict of FAA regulatory enforcement and company disciplinary action with the safety purposes of these programs — leading to deviations from the memorandum of understanding (MOU) and reduced effectiveness — and what today seem to be excessive levels of data protection given the priority of system safety analysis.
“All these programs were created independently,” he said. “We have different analysis. We have different taxonomies, meaning [that] one ASAP program might be running on a different software with different taxonomies than the other. So, to mix a flight attendant ASAP report and a pilot ASAP report, it can be challenging and doesn’t necessarily equate very easily.”
Different governance documents for each voluntary safety program also complicate efforts by airlines to build holistic views of problems. “The flight attendants, the mechanics, the dispatchers and the pilots all have a separate MOU,” he said. “They don’t have to coordinate, and they don’t have to talk. … None of that guidance has ever been coordinated with regard to how we are going to use [these programs] systemically from a safety management standpoint.”
The practical, efficiency-related consequences have included duplication of staff and resources — people doing the same job with a silo mentality in operational departments and employee work groups. During safety data analysis, the airline often cannot “take one report and overlay it, [we] can’t take the FOQA data and mix it with the other data also,” Logan said.
Another complication has been disparate treatment of some airline employees — such as ramp operations staff — because they are not covered by an ASAP program. Moreover, the defensive, adversarial mindset from the time when voluntary safety programs were designed has become an anachronism, he added.
“All of the guidance that we used … was created [or] came out of the existing FAA enforcement handbooks and enforcement guidance,” Logan said. “There was a need for these [provisions] to be put in place at the time they were implemented … We had no strategic vision [of] what we wanted to do with this data once we had it, and how we were going to fit it in [to SMS]. …This is just the reality of how we got these programs going.”
New Imperatives
People who helped to develop these programs — Logan included — especially did not have today’s prevailing concept of how to use safety information that inherently would identify human errors, he now believes. “[Safety] is a human process; humans make errors,” Logan said. “A lot of those errors are built into the system, and they’re caused by the system or facilitated by the system. We do not deal with that well. We still look at the violation aspect of [events] first, and deal with that without really understanding it’s an error, it’s human, we need to figure out how we are going to fix it.”
Part of the core issue is the mindset of people who dismiss voluntary safety programs as merely a way to “get out of jail free,” that is, avoid disciplinary action.
He added, “I will passionately disagree; in the majority of the cases, that is not the case. We do much more proactive work with our employees who report into the program than is ever evident. … We make sure that [the reporting] employee understands what they should have been doing, gets trained on those issues they didn’t understand — even up to going into the simulator or going out and demonstrating [competence]. That is the best proactive safety you can do, and it does fix things.”
‘Rogue Employee’ Myth
During the design of FOQA and ASAP programs, participants sometimes voiced a need to find rogue employees, those whose unsafe acts would be considered intentional. Some participants would say, “We all know they’re out there,” he recalled. “At the time, the lack of trust and history probably made these possibilities believable.”
The past and present workplace cultures of voluntary safety programs differ, however. “We have to understand that these were created in an environment where there was little or no trust,” Logan said. “We were back in the days [in the 1980s and 1990s] where it was ‘Catch me if you can.’ … The value of the information [now] has largely overcome the lack of trust between the parties.”
As a result, in almost 20 years, U.S. airlines typically have not identified rogue employees. “At the [U.S. Federal Aviation Regulations Part] 121 carrier level, our data does not identify rogue employees [abusing the reporting incentives],” he said. “It really identifies system safety issues that we have to deal with — whether it’s a training issue, whether it’s just information to the crews, or whether … we didn’t do a good job when we set up our process in the airlines, whether it’s a procedure, a checklist or any [other] items.”
Similar concerns shaped the original relationship between the government and airlines, he said. “What was going to happen if [airlines showed] this information to the FAA or to the [U.S. National Transportation Safety Board (NTSB)] or to the Congress?” Logan recalled. “Would we be [identified publicly] as a rogue carrier or a rogue operation? That never happened, but we were concerned about it, so we operated as if it was going to happen.”
Questionable Barriers
In today’s airline SMSs, the legacy of suspicion about voluntary safety programs has been detrimental, impeding full SMS effectiveness — in his experience. “What happened is we have a limited ability to cross-pollinate the [ASAP] information because of the data protections that have been put in place,” Logan said. “We lose pertinent [FOQA] information because some of the information goes away after a certain amount of time. … [We’ve] put controls into the software that delete the flight number and the date, so we may not necessarily know that information [that is, identification of individual flight segments is kept secret, allowing only aggregate analysis of multiple flights]. There is a barrier to systematic safety analysis. And those barriers prevent us from smoothly evolving an accident [or] an incident investigation.”
Limitations on how Southwest Airlines can analyze runway excursions versus near-excursions illustrate the point. “If I have an airplane that slides off a runway, and I know about it, I can pull a flight [data] recorder; I can grab the [cockpit voice recorder data],” Logan said. “I can work with the NTSB on those issues [see “NTSB Seeks Data for Safety Recommendations”]. I can pull all the information I want — immediately — off of the airplane. … I can bring the crew in and do an interview. I can do everything I want to do.”
Conversely, if he receives a report from a captain or other party who reports — through one of the voluntary safety programs — that an aircraft “almost” ran off a runway, the analysis runs into the barriers. “If [the report] comes in through one of the programs, the ability to do that same investigation is limited, and I’m not sure that’s in our best interest, especially when we look [from] an SMS standpoint,” Logan said. “The idea [in SMS] is we are supposed to be using those near-miss events to be able to get ahead of those events. If we can get at the data [yet] we can’t use the data, we’re not going to be able to do that. … In a perfect world, we could use the same techniques we have developed for accident investigations, such as flight data information [FOQA], crew interviews [ASAPs] and review of associated data in the investigation to identify hazards.”
Another example cited is monitoring system safety whenever Southwest Airlines begins flying into an airport that is new for the company. “We’re going to [need to] get information that’s going to tell us something that we may not have known,” he said. “We have to be able to react to that.”
Airline LOSA programs and the FAA’s Aviation Safety Information Analysis and Sharing (ASIAS) program encounter similar aggregated-data access/use limitations and missing context.
Worldwide fleet data collected by airplane manufacturers also have been a missing source of information in airline SMSs, Logan said. “We need to have that part of it if we are going to do [SMS] systems analysis on our issues,” he said. “There are fleet-specific or national airspace issues that we, in the carriers, can’t alone really move or push. We’re trying to do that through ASIAS, [and] through the [U.S. Commercial Aviation Safety Team] process, but it’s not as easy.”
Call for Reassessment
Recommendations in the presentation aim to resolve the SMS impediments described. “The FAA administrator needs to establish an aviation rulemaking committee … to get the industry, [government, academia,] labor and other folks together to really review all of the guidance that’s out there on these voluntary safety programs and integrate them with SMS,” Logan said. “When you look at the SMS guidance, you’ll see references to the voluntary safety program[s] but there is no tie [from SMS] across to those programs. … Let’s write [new] guidance to match up to what we need from SMS; let’s not do it from the bottom up. We [also] need to make sure that the Part 193 [U.S. Freedom of Information Act] protections are embedded [explicitly within FAA-accepted SMS programs and apply to] any of the information [an analyst in] SMS comes across … that’s the best way to make sure that the information will be available.”
New educational programs about SMS could remove some secrecy surrounding airline safety activities, which prevents elected officials and the public from understanding the nature and results of SMS. “The problem is nobody outside of our group really knows how much we’re doing, and nobody has really studied it,” he said. “I think the only way to do it is to start the conversation. I would really like to see us come together, and maybe ISASI is the perfect place to do that, or maybe Flight Safety Foundation is the perfect place to do that. … … [We] have to lift the veil a little bit and say [publicly] ‘Know what? We do have [safety] events out there, but we deal with them very well.’ We usually come up with corrective actions very [quickly and methodically] and put implementation [of solutions] in place. … Barriers to data usage and data correlation should be eliminated.”
In the question-and-answer session, Mont Smith, director, safety, Airlines for America, said that twice-a-year, government-industry Aviation Safety InfoShare meetings perhaps should provide educational content for selected observers who could communicate about safety accomplishments of ASIAS and the airlines. Logan agreed.