Startle Update

For at least a decade, a steady rise in disengagements of automatic flight control systems on airliners — often minutes before accidents or incidents — has captured the attention of aviation safety scientists in Europe. Among research paths being taken against the backdrop of loss of control–in flight (LOC-I) are suspected causal factors such as pilots’ startle response, their automation overreliance, cognitive overload, and limited or inadequate manual flying skills to operate the current generation of commercial transport aircraft.

In April, one group reported its first insights and preliminary implications from a multidimensional project launched in 2012 and running through the end of 2015, called Manual Operations for 4th Generation Airliners (MAN4GEN), funded by the European Union. This international consortium — led by the Netherlands National Aerospace Laboratory (NLR) and the German Aerospace Center (DLR) — gathered subject matter specialists from Airbus, Boeing, Global Training Aviation, International Development of Technology (IDT), Sweden’s Linköping University, and Austria’s Medical University of Vienna and Vienna University.

“Despite the substantial and proven safety benefits of automation systems in third- and fourth-generation aircraft, evidence indicates that when faced with unexpected and challenging situations, pilots sometimes have difficulties in quickly responding to situations which require a rapid transition in their activity from monitors of very reliable systems, to active and authoritative decision makers exercising manual control of the aircraft,” the consortium’s mission statement says in part.

“Without a set of [piloting] skills that also include the ability to manually control an aircraft, to manage the automation systems effectively, [to] always maintain an acceptable level of situational awareness, and [to] remain in control of the aircraft, these accidents will continue to occur. Such instances have occurred in the highly augmented fourth-generation aircraft, as well as conventional aircraft, and with experienced pilots fully trained to current standards. These accidents are often due to a combination of the crew not managing the aircraft systems effectively after an unexpected event, and being unable to apply appropriate manual handling skills. MAN4GEN aims to identify the common thread behind the events that lead to these accidents, and to recommend short-term changes to operational procedures, training and aircraft systems technology in order to mitigate this threat to aviation safety.”

The consortium partners have pursued a deeper understanding of the root causes, including contributory human factors, of losing situational awareness in highly augmented and automated aircraft, says Sunjoo Advani, an aerospace engineer and owner of IDT, co-author of the grant proposal and MAN4GEN partner. The desirability of research in these areas had been discussed during five years of work on LOC-I mitigations by the Royal Aeronautical Society–sponsored International Committee for Aviation Training in Extended Envelopes (ICATEE), which he chaired (ASW, 7–8/14).

“The training of pilots on the automation, the failures that we could have in the automation, depends on what we have in the simulators,” Advani said during a presentation to the World Aviation Training Conference and Tradeshow in Orlando, Florida, U.S. “Usually, we do not train automation failures in our devices although crews are trained to rely on automation and envelope protection. Automation is either on or off, but slight failures or slight deviations are usually not trained, they’re not programmed in there. And so we’re very limited in assimilating the unusual behavior of the automation.”

Automation systems in some circumstances can take the flight crew “out of the loop” of airplane control — also called becoming dissociated — so he said the challenge is finding the right balance of behaviors required to be “in the loop/monitoring” or getting “back in the loop” while at all times flying the airplane.

“A loss of control situation can also be a psychological loss of control, a loss of situational awareness and the inability to come back into the loop,” Advani told AeroSafety World, reiterating the importance of early identification of subtle deviations. “As the deviation slowly starts to occur, then the crew must react to that at the right time. If it’s a prolonged or a delayed reaction, that can get them into trouble and may become the cause of the startle. If we can figure out what causes the startle before it actually occurs, what causes confusion in the pilot’s mind, maybe then we can cover these factors in training. This project is teaching us more than finding just one solution to one problem in one event. We’re starting to learn better the process of designing scenarios so that we evaluate pilot competencies.”

Preliminary Findings

Advani’s preview of findings of MAN4GEN came from an experiment conducted in two research flight simulators. The experiment was designed to investigate the tendency of some qualified and current airline pilots — while monitoring a highly reliable flight control system — “to have difficulties with interpreting the situation, managing the flight path and making decisions with effects on the ability to maintain a safe aircraft state, reacting appropriately to unexpected events,” he said.

The experiment in these simulators fell within a broader MAN4GEN work package structure principally involving analysis of the world’s relevant academic and government studies, evaluation of pilot performance in a test-aircraft environment under operational conditions, and brain-imaging studies of pilots “to investigate the loss of situational awareness from a neurological perspective.”

The consortium’s website notes that the simulator experiments included in the research program looked at aspects of flight crews’ management of serious in-flight events, especially “elements of ‘startle,’ unexpected or even unforeseen occurrences, whilst coping with stressful situations (including physiological and/or emotional responses),” as well as how to physically and psychologically regain control of the aircraft in such circumstances. “The outcomes of MAN4GEN will be presented to industry and the research communities,” the website added.

Next steps will involve short-term and medium-term development of mitigation concepts for the application of scientifically robust procedures, training and technology. Of special interest to the researchers as of early 2015 is the prospect of introducing training scenarios and validated teaching methods through new guidance material coordinated within the International Civil Aviation Organization’s (ICAO’s) global standards as done when ICATEE drafted recommendations for the mitigation of LOC-I that ICAO approved and published in 2014.

Regarding the simulator experiment’s first set of scenario-based tests, while a second set of tests continued as of April, “The intention of our experiment was to study how the crews’ decision making and the risk assessment occur in response to an unexpected event,” he said. “So we looked at how and when they revert to manual control, and how we can force the reversion to manual control in unexpected and challenging situations. To assess active, authoritative decision making,  we observed the crews using various metrics.”

First Experiment

The first set of tests was conducted in NLR’s GRACE simulator in Amsterdam configured as a Boeing 747-400, and in DLR’s AVES simulator configured as an Airbus A320 in Braunschweig. Twelve crews flew the same scenario (Figure 1) based on a limited knowledge of what to expect that was strictly controlled by the researchers.

“We told the crews, ‘You’re making an approach, and you have a low fuel state, not extremely low but enough fuel to fly to your alternate and conduct one approach. Visibility is 20 km [12 mi] as you start off. Cloud bases are at 2,000 ft.’  There was also a thunderstorm moving away from the airport vicinity.”

Because of sudden limited visibility or the wind shift, all the crews conducted a missed approach. Each crew’s decision, however, triggered a series of events not briefed before the scenario. As soon as air traffic control (ATC) issued a vector to a new heading, the heading-select function of the simulator’s automated flight control system failed. As the pilot flying dialed the new heading, nothing happened on the mode control panel or primary flight display. To turn to the new heading, each crew had to revert to manual control.

Upon selecting the autopilot OFF, a bird strike occurred. “Engine one fails, engines three and four are surging so they’re creating a strange kind of a sound because they’re on a high power setting. If the crew reduced the power, they noticed that the sound would go away and the surging would actually stop. Crews were given vectoring to holding, and the visibility cleared up,” he said. Their next step was to decide whether to go back to the same runway, to go to an alternate airport or land on a suitable departure airport runway in a different direction.

Researchers directly observed each crew’s behavior from the instructor’s point of view. They also studied video recordings, recorded interviews after the simulator session, responses to questionnaires, a crew debriefing, recorded simulator data, and each pilot’s heart rate monitoring and eye-tracking.

“The distilled procedure they’re actually using day-to-day in routine operations says to identify and confirm the problem. Perform recalled memory items. Create a short-term plan,” Advani said, describing what researchers expected to see. They expected concentration on performing high-priority tasks that must be done right away, most importantly managing the flight path but then communicating with ATC. “Then they go through the normal and emergency checklists. Then they create a long-term plan including checking airplane controllability and performance,” he said.

Advani summarized what were deemed significant deviations of the crews from these expectations. “The crews indeed experienced elements of the scenario as an unexpected event,” he said. “They were very surprised and they appeared to have more difficulty with the scenario than we had ever expected. We thought it would really be a piece of cake for them to deal with this.

“So in some cases, this even led to unstable approaches and very short final lining-up distances from the runway. Surprisingly, just a very slight change in the way that we set up these sequences led to problems. The decision by some to land as quickly as possible led to an abbreviated set of procedures and checklists, if run it all. Some crews failed to perform the threat assessment and made decisions without considering the impact.”

In the 747 scenario at NLR, 67 percent of the crews performed the “multiple-engine flameout or stall” procedure despite training that called for performing the “engine limit or surge or stall” procedure.

“More than 50 percent returned to the same runway even though there were weather difficulties there just a few minutes ago,” Advani said. “The wind had shifted, and still they were selecting a downwind runway. Why? I don’t know. Other possible available procedures were not used. There was a lack of energy management that we noticed in most of the crews, and the leadership was not often done in an optimal way.”

In the comparable A320 scenario at DLR, most of the eight crews immediately identified their situation as an emergency, and they returned to land as quickly as possible. “They didn’t have a complete failure in either engine but had the same stall-surging occurring,” he said. “The majority of the crews identified the double engine stall but did not perform the ‘engine limit or surge or stall’ procedure. Some crews did not complete the one-engine stall checklist and even forgot the landing checklist for the second approach. Six crews returned to the takeoff runway or an adjacent parallel runway even though they could expect a tail wind, and only three of the crews stabilized the engines. Two decided to fly on idle thrust for the complete flight after the bird strike.”

For the researchers, one issue was how to interpret pilots’ behaviors from situational awareness and sense-making points of view. Observers saw indicators of increased workload and reduced situational awareness after the bird strike. “The pilot flying typically was very occupied with flying and potentially overloaded, so the procedures weren’t used.” Advani said. “Very few crews, as in the 747 simulator, consciously dealt with the energy management. Eventually, all of them, fortunately, managed to land the aircraft. The crews with good workload management and decision making stood out very obviously. Decisions were made by others without the consideration of the additional risks and the possible consequences.”

Summarizing the project’s status, he said the MAN4GEN partners expect to generate explanations and clarifications of what was recorded and measured. “Flight crews appeared insufficiently trained for an atypical scenario so we need to train resilience,” he said. Regarding flexibility needed, he noted that the researchers suggested — but did not require — that pilots keep the designated pilot flying/pilot monitoring roles in which they began the simulator experiment. Outcomes might have been better, however, if pilots had changed roles at the outset to match their abilities and experience during the cascading surprises and unusual failures. “There was no clear-cut answer to this,” he said. “And the crew-observation methods that we’ve been looking at could help the industry to better assess the training. It’s really important and very interesting how scenario design can influence the outcome. So we need to work together as an industry to improve the training for unexpected situations — through effective scenario design, instruction and debriefing.”

Featured image: © karmagarda | iStockphoto
Pilot selecting control: © Oleg Belyakov | AirTeamImages