Virtually Interactive

Efforts to computer-generate air traffic control (ATC) environments inside flight simulation training devices (FSTDs) have advanced significantly in step with an emerging consensus about the benefits, several specialists say. The airline industry has spent about six years considering systems and methods that would go far beyond current training of candidates for the multi-crew pilot license (MPL) in a few countries. Panelists spoke in April during the World Aviation Training Conference and Trade Show (WATS 2010) in Orlando, Florida, U.S.

Other conference speakers urged caution in deploying the emerging capabilities, saying that disruptive effects on primary training objectives for experienced airline pilots ultimately could outweigh the safety benefits of added realism. Synthesized interactive ATC radio communication also might complicate an already rapid proliferation of special purpose operational training, said Rory Kay, executive air safety chairman, Air Line Pilots Association, International (ALPA), and a United Airlines captain.

“Sessions increasingly are crammed with mandatory training and checking of maneuvers such as [those for] controlled flight into terrain, traffic-alert and collision avoidance system, head-up display, Category III autoland, wind shear, required navigation performance–area navigation and [airplane] upset recovery, and the list will keep getting longer,” Kay said. “We need more time, not less, in training scenarios to truly practice basics, and to be truly trained to proficiency.”

Proponents of ATC simulation in FSTDs have stressed that in environments of high-density traffic control, airline flight crews’ attention unavoidably becomes divided between flying the aircraft and listening to radio communication for the flight’s call sign, but airline pilot training standards have yet to formally recognize the corresponding need for realistic ATC communication in FSTDs.

“We wouldn’t be in the golden age of safety without going hand-in-hand with the golden age of automation,” U.S. Federal Aviation (FAA) Administrator Randy Babbitt told the attendees. “We have far greater capabilities today to replicate almost anything in an aircraft, and to expose people not in harm’s way but in the educational way to situations that will confront them as crewmembers. We can replicate every scenario [with] wonderful positive teaching tools. We should maximize simulation usage.”

Implementing ATC simulation in FSTDs involves human factors issues and technical challenges that the industry has not faced previously, said Nassima Hamza, business development manager, Thales Training and Simulation. “Delivering a user-friendly, robust system that eventually will try to simulate a human being is not an easy task — especially when it has to interact intelligently with the crew on the flight deck by means of speech recognition and, at the same time, be coherent and correlated with the rest of the cues provided by the other simulator subsystems.”

Last year, Hamza coordinated an international industry survey sponsored by the Royal Aeronautical Society and Halldale Media Group, asking pilot-training professionals to identify safety objectives for improving pilots’ radiotelephony skills in relation to ATC and the roles they see for ATC simulation.¹

“The majority of respondents said they had never used an FSTD fitted with ATC simulation, and they agreed it is a missing link,” Hamza said. “Interestingly, training professionals and regulators each have a different opinion on the efficiency of instructor role-play as an acceptable means of compliance. Most respondents see benefit in using ATC simulation outside the scope of the MPL, but some concerns were expressed over its use for training experienced crews.”

The respondents’ priorities were: developing situational awareness in a realistic environment with audible and visible air/ground traffic, generating interactive communication with ATC and correct aircraft locations, representing ATC communication workload in connection with a virtual controller, strictly using International Civil Aviation Organization (ICAO) phraseology, developing threat and error management skills through scenario-based training, and enhancing English skills, especially for pilots from non-English-speaking countries, she said.

The forthcoming adoption of ICAO Doc 9625, Manual of Criteria for the Qualification of Flight Simulation Training Devices,² by national aviation authorities will push further the need for acceptable systems. “The first key is for all stakeholders to carry on defining the ATC simulation solution that would enhance the learning experience without compromising or conflicting with the prime flight training objective,” she said.

Future flight crew licensing regulations of the European Aviation Safety Agency (EASA) are expected to incorporate this concept, said Marsha Bell, vice president, Commercial Pilot Training and Systems, Adacel, and panel moderator. “ICAO Annex 1, Personnel Licensing, and PANS-OPS Training are not explicitly requiring ATC environment simulation,” she added. “Authorities around the world are allowing for other means of compliance, like flights in the cockpit of the MPL candidate’s future airline. This will be the case for another two to three years.”

Safety Benefit Questions

One conference speaker said the industry has yet to take full advantage of FSTDs. “The question is, ‘Have we really increased flight safety by harnessing new flight simulation technology?’” said Kip Caudrey, senior manager, simulator evaluation, standards and regulatory affairs for Boeing Training. “There are things that could be done in a flight simulator these days that we are not doing, or that we could do better.”

He included air traffic controller and air/ground traffic simulation to a list of potential FSTD-based improvements comprising stall recognition and recovery training, upset recovery training, storm front avoidance procedures, unavoidable thunderstorm entry, runway incursion avoidance, realistic landing training, special airport training, and volcanic ash encounters. “Each one of these would significantly improve flight safety,” Caudrey said.

“Anyone involved with ATC simulation [already has] an appreciation of the runway incursion challenge,” he said. “With additional aircraft over the next 20 years … the greatest risk is runway incursions [involving] pilots not understanding taxi instructions. … Much as we also would like to say there is standard [pilot-ATC phraseology everywhere], pilots know that, unfortunately, this is just not true. … It is going to be a long time before we can really replicate what goes on in the real world.”

Text-to-Speech

Text-to-speech (TTS), speech recognition and speech synthesis constantly are evolving and already are more advanced than the airline community may realize, said panelist Marc Fabiani, product manager, Network TTS, Nuance Communications.

“TTS is at a very high level, and the output is rather natural and can be indistinguishable from human speech,” Fabiani said, playing audio files of speech generation with several synthetic voices. “In terms of accuracy … the speech actually can be better than a human being’s because we can leverage [computer] intelligence in terms of how abbreviations, [verbal] shorthand and pronunciations work. The technology can encode much more knowledge and be a lot more accurate than one human being.” The text in–speech out capability has become easy to use; software programmers send the script from a dialogue or other textual information and the speech is audio-streamed or saved to a computer audio file for further processing, Fabiani said.

Several speech-processing challenges remain, however, for handling interaction of synthetic air traffic controllers and human pilots. “We do not have the voice variety that many [system engineers] would need to be able to emulate multiple users, such as multiple pilots or [controllers]. We offer at best a half dozen voices per language.

“Another issue is expressivity. We can select different moods out of the [TTS processing] engine, but we cannot expect the TTS engine to act on its own … understanding the whole script and pronouncing [responses] in a certain way based on that information. It does not have that artificial intelligence capability yet. It lacks the full spectrum of emotions, so it can ‘speak’ with urgency or passion but not panic or [humor].”

Expanding Interest

U.S. research on improving training for pilot-ATC radio communication lately has been driven by the FAA’s Advanced Qualification Program (AQP), based on data from line-oriented flight training (LOFT) and initial operating experience (IOE), said Judith Bürki-Cohen, principal investigator, Flight Simulator Human Factors Program, U.S. National Transportation Systems Center.

“The Next Generation Air Transportation System [NextGen] for the United States will affect pilot-ATC communications training,” she said. “This will involve transitioning from primarily voice to primarily data communications. … Much tighter communication and automation will be one additional factor.”

When they first arrived for IOE, some airline pilots observed in her studies seemed to experience difficulty dealing with ATC in high-density airspace, Bürki-Cohen said. A second study attempted to validate the earlier observations by analysis of the U.S. National Aeronautics and Space Administration (NASA) Aviation Safety Reporting System (ASRS) database for relevant reports involving IOE and radio communication, and 93 errors were found.

“The majority were altitude and crossing restriction violations, but we even have had unauthorized takeoffs and landings on the wrong runway and even at the wrong airport, etc.,” she said. “Some of these issues may be alleviated and some issues will not be alleviated by data communication [datacomm] — meaning data-linked textual and visual information. This information does not disappear, and the pilots can read it when they are ready. Some clearances may even be uploaded directly to flight management systems. At the same time, with datacomm, pilots will not hear the urgency in the controller’s voice.”

Datacomm also is expected to reduce misunderstandings caused by pilots’ and controllers’ accents, speech rates and culturally different intonations, especially when pilots operate into airports where languages are foreign to them, she said. “The [datacomm] challenge will be an enormous increase in head-down time,” Bürki-Cohen said. “The pilots also will lose the party line, and this will affect not only the situational awareness of the pilots but also that of controllers, because pilots may ask more questions about weather and traffic information, information that otherwise they would have gleaned from the party line or from other aircraft. Also, datacomm readback by the pilot is passive — just a button push — so the controller [may wonder] ‘Does this pilot really understand what I mean?’ The controllers get no information from the intonation, the hesitation or perhaps the emotion of the pilot that they can hear in the voice.”

For the foreseeable future, both professions will interact in a mixed voice and datacomm environment, she said. ATC communication requiring immediate response, such as many clearance instructions, still will be delivered by voice.

Some specialists have concerns that attention to aircraft call signs and recognition of urgency in voices could diminish as flight crews transition primarily to visually displayed information in NextGen. “Therefore, simulation of radio communication is very important for safety, and likely will remain so for clearances requiring immediate action,” Bürki-Cohen said.

Role-Playing Experience

Although ATC simulation for FSTDs initially was driven by MPL requirements, the airline industry should expect expansion to other types of ab initio training “where the pilot’s ATC communication skill set is not going to be acquired by exposure — time in aircraft — but by the synthetic environment of the simulator,” said Bryan Burks, vice chair, Training Council, ALPA, and an Alaska Airlines captain.

ICAO’s International Working Group on FSTDs spent about three years addressing what it deemed a clear training need for this emerging technology. “The interesting part will be the interplay as the industry introduces simulated ATC environments into training for pilots other than MPL candidates or brand-new, zero-time ab initio pilots,” said Burks, a working group member. “That remains a challenge … one that ALPA is looking at based on a data-­driven approach. If it works and it doesn’t impede or harm the training objective, we look forward to incorporating that technology … into other training activities for more mature pilots, recurrent training or type-rating training. Right now, it is still in beta test and its appropriate place … is in MPL or ab initio training.”

As in the case of datacomm, ALPA expects the technology itself to introduce problems. “Where we take a conservative approach is crews [receiving] recurrent training or type rating training, with training objectives defined according to written performance standards that usually don’t involve ATC interaction,” Burks said. “So first, do no harm.”

The primary benefits for other airline pilots likely will be indirect. “This would unload the instructors and evaluators, who often use scripted types of ATC communications with the pilots,” Burks said. “It would allow them to focus on evaluation better than when role-playing ATC.” A psychological component is that, when pilots go into the flight simulator for a checking or evaluation event, they may perform differently for an examiner or check airman issuing ATC instructions than they would if a real controller were issuing the instructions. “If we had a technology that could be a truer pilot-ATC interface, we could take out this artificiality,” he noted.

The flight crew–ATC interaction is especially important when conducting FSTD training to mitigate specific threats such as unstabilized approaches involving flight crew compliance with unsafe ATC instructions. “Sometimes ATC is a threat that the flight crew has to manage,” Burks said. “Through the instructor-led role-playing of ATC, we might not reach the desired objective if it involves ATC-crew interplay during an incident or accident scenario.”

U.S. Regulatory Perspective

Human speech emulation and artificial intelligence powerful enough to enhance ATC realism in FSTDs have been discussed by government and industry for several years, said Mike Wilson, aviation safety inspector, Air Carrier Training Branch, FAA.

“The FAA and other regulators must make sure that we are continuing to capitalize on crew training and not only maximize, but require [flight] simulation,” Wilson said. “We want more effective training, not just more training.” External pressures keep building to accomplish ever more training objectives in simulator sessions, he said.

“The variety of new technologies about to come into the stream of pilot training — datacomm, required navigation performance, enhanced flight visual systems — all require new phraseology, new terminology and new acronyms that have to be addressed,” Wilson said. TTS technology may provide more flexibility in training, but regulation writing likely will have to follow industry consensus about a sound basis for any mandatory changes, he added.

“Right now, there is no FAA requirement [for simulation of ATC environment] although Doc 9625 incorporates that as a task requirement,” Wilson said. “To adopt it from the FAA side, we need to have a discussion to determine what kind of requirement is necessary. ATC environment training [already] is a part of every pilot’s training; it’s just completed now in a different way —without the new technology. New FSTD certification would not be required because ATC environment simulations would not fit the criteria of an aircraft safety of flight issue.”

“In an environment of changes [to the National Airspace System] on an almost daily or monthly basis, we need the flexibility of TTS so that we can incorporate some of the new [avionics] boxes that create a need for further pilot training. Our overarching goal in the last six years has been to allow for this technology to grow.”

Notes

1. Royal Aeronautical Society; Halldale Media Group. “Industry Consultation: ATC Simulation in Flight Crew Training.” November 2009.
2. ICAO. Manual of Criteria for the Qualification of Flight Simulation Training Devices. Doc 9625, Third Edition, 2009.

Download the PDF of this article.