Although cockpit automation is intended partly to give pilots more time to think about — and plan for — upcoming portions of a flight, the pilots may not be devoting all of that time to flight-related thoughts, a new study has found.
Some past studies have shown that, when pilots were queried while using automated cockpits, they sometimes were unable to answer basic questions or even to tell the questioner where they were.
Other studies showed that automation “can sometimes relieve pilots of tedious control tasks and afford them more time to think ahead,” said the report on the new study, conducted by Stephen M. Casner of the U.S. National Aeronautics and Space Administration Ames Research Center and Jonathan W. Schooler of the University of California, Santa Barbara. “Paradoxically, automation has also been shown to lead to lesser awareness. These results prompt the question of what pilots think about while using automation.”
To find out, Casner and Schooler designed a test, conducted in a Boeing 747-400 simulator, in which 18 experienced pilots flew an arrival procedure to New York’s Kennedy International Airport (JFK). The 747-400 “practically offers the pilot two levels of automation” for an arrival procedure, and the instructions to the participating pilots were to “fly as they normally would, deciding to use the automation as they saw fit,” the report said.
As the pilots flew, they were asked periodically what they were thinking — “whether their thoughts were directed at the task at hand; if they were thinking higher-order thoughts about the flight, such as planning ahead; or were thinking about something entirely unrelated to the flight.”
In addition, their use of automation and other aspects of their performance were recorded.
Waypoints
The arrival procedure incorporates a series of six waypoints — predetermined geographical positions defined by latitude-longtiude coordinates, in this case associated with a VHF omnidirectional radio (VOR) station — that serve as landmarks on the arrival path to JFK (Figure 1). The researchers were especially interested in how the pilots handled the two waypoints designated as crossing restrictions — HARTY, which they were required to cross at 16,000 ft, and LENDY, which required an assigned altitude of 6,000 ft and an airspeed of 250 kt. Acceptable performance, as defined by the U.S. Federal Aviation Administration was to pass within 1 nm (2 km) of the waypoints and to be at the assigned airspeed, plus or minus 10 kt.
“To achieve the two prescribed altitudes associated with the HARTY and LENDY waypoints and the assigned speed at the LENDY waypoint, the flight crew has a choice between two levels of automation,” the report said. “The difference between these two levels of automation lies in how much of the work of following the route the flight crew wishes to hand over to the flight management computer and how much they would like to perform themselves.”
If the participating pilots wanted the higher level of automation, they engaged the vertical navigation (VNAV) function for help in meeting altitude and speed restrictions associated with the waypoints.
“The VNAV function performs a fairly complex task,” the report said. “The VNAV function must decide when to commence a descent to obey the altitudes associated with the two waypoints. In this case, if the airplane is cruising at 39,000 ft, then the airplane must descend a total of 23,000 ft. If the computer chooses a descent rate of 2,000 fpm, then the descent will take 11.5 min to complete. If the airplane is traveling at a speed of 300 kt, then it traverses 5.5 nm [10.2 km] per minute. During the 11.5 minutes needed to complete the descent, the airplane will traverse 63.25 nm [117.14 km]. The computer concluded that the descent must be commenced 63.25 nm prior to HARTY.”
The flight management computer also can determine the best way of reducing airspeed from 300 kt to the 250 kt required to cross LENDY, the report said, adding that while computer functions calculate and implement the required actions, the flight crew must monitor the airplane’s progress and be prepared to intervene if they believe the airplane will not achieve the required altitudes and airspeed.
For the lower level of automation, pilots engaged the lateral navigation function for automation of navigation between waypoints, but they retained more responsibility for meeting the altitude and airspeed requirements.
“Pilots’ choices about which level of automation to use are often more nuanced than it might seem,” the report said, noting that to use the VNAV function, pilots need sufficient time not only to enter initial altitude and airspeed information into the flight management computer but also to reprogram the device in the event that air traffic control (ATC) instructions or weather conditions require changes.
Routine Flights
The participating pilots — nine captains and nine first officers with an average of more than 11,000 flight hours, 69 percent of which, on average, was spent in advanced cockpits — were paired up to fly the arrival procedure, and each pilot took a turn at the controls. They were told to fly as they would during any routine flight, using whatever automation functions they wanted.
A researcher seated behind them read the ATC clearance and every two minutes — while noting the automation functions in use and whether the pilot was at that moment interacting with the flight management computer by keypad or by the mode control panel — prompted the pilots to characterize the nature of their thoughts by categories. (A response of “1” meant that a pilot was thinking about a “task at hand”; a “2” meant the pilot was thinking about “something related to the flight but not something that was happening in front of them at that moment”; and a “3” meant the pilot’s thoughts were “not related to the flight,” the report said.) A flight data recorder noted the altitude and airspeed when the pilots crossed the HARTY and LENDY waypoints.
Thought Categories
The pilots were asked about their thoughts eight times each — a total of 144 times. Table 1 shows the thought categories and the relationship between the automation level and the pilots’ thoughts.
Differences between the top row and the bottom row show what the researchers described as “a significant shift from task-at-hand thoughts (1s) to higher-level thoughts about the flight (2s) when the higher level of automation was used. … This finding supports … what is perhaps the most closely held belief about automation: that the use of a higher level of automation is associated with pilots thinking fewer task-at-hand thoughts (1s) and more higher-level thoughts about the flight (2s).”
Thoughts and Successful Piloting
Table 2 shows the relationship between what the pilots were thinking and the extent of their success in meeting the two crossing restrictions within 300 ft of the assigned altitude and 10 kt of the assigned airspeed.
“Pilots’ thoughts did not drift onto other [non-flight-related] topics when a higher level of automation was used but rather when either level of automation was used successfully,” the report said. “Difficulties in meeting the crossing restrictions were associated with pilots reporting more task-at-hand thoughts (1s), suggesting that automation struggles diverted pilots’ attention away from higher-level thoughts about the flight (2s). However, when pilots enjoyed more success and reported fewer task-at-hand thoughts (1s), their thoughts seemed to move on to task-unrelated topics (3s).”
Table 3 shows the relationship between automation interactions and pilot thought categories. “As was the case with the success variable, interacting with the automation was not associated with fewer higher-level thoughts about the flight (2s),” the report said. “It was associated with fewer task-unrelated thoughts (3s).”
Reconciling Basic Claims
The researchers said that their results “help reconcile some of the most basic and seemingly contradictory claims about the effect of automation on pilots’ thinking.”
On one hand, the report said, “Our data support the most closely held belief about automation — that the use of more automation allows pilots to engage in fewer task-at-hand thoughts and more higher-level thoughts about the flight.”
However, the data also “were consistent with studies that demonstrate that, when more automation is used, measures of pilot awareness show that less, not more, higher-level, flight-related thinking has taken place,” the report added. “When difficulties were encountered, the use of a higher level of automation may have substituted one sort of attention-demanding work for another. When all was going to plan and the task of managing the airplane was seemingly under control, pilots often opted to think about something else.”
‘Advantages and Disadvantages’
The report cited earlier studies that have found that people spend about 30 percent of their waking lives in task-unrelated thought — an activity that the authors said is “nuanced with advantages and disadvantages.” They noted that the pilots who participated in the study were recorded as spending about the same proportion of their time in task-unrelated thought, also called mind-wandering.
The report also cited earlier studies that indicated that mind-wandering might be a critical element of human cognition with several important functions, such as providing a mental break that enables individuals to return to work with “improvements in vigilance performance” or boosting problem-solving skills. Another study says that task-unrelated though enables “autobiographical planning,” the report said, adding that pilots who engage in autobiographical planning “may be thinking ahead not only to future portions of a flight but rather to future portions of their life.”
Other studies demonstrated disadvantages associated with mind-wandering, including a greater tendency to commit errors, a predictable decline in reading comprehension and a “more careless response in a go/no-go decision task,” the report said.
Cockpit Design
Overall, the report by Casner and Schooler said, “We must consider the possibility that the thought patterns we observe among pilots are the rational outcome of the way we have designed cockpit automation systems.”
Because automation handles so much of the work, “we may have left pilots with little incentive to think beyond the steps needed to configure the automation and the aircraft behaviors that these steps produce,” the report said. “And since pilots receive little procedural guidance about how to actively monitor automated systems, we may have effectively left them with the question ‘What else is there to think about?’ If this is the case, we might wonder if we could encourage a different use of pilots’ mental free time.”
The report cited earlier proposals by other researchers that called for development of specific procedures for monitoring in an automated cockpit or for the design of automated systems that require periodic action by pilots, even when the flight is running smoothly. A 2006 study by one of the authors found that “even perfunctory conversation among pilots about where they were and where they were going was enough to reverse the ‘out-of-the-loop’ effects seemingly caused by using advanced navigational automation,” the report said.
This article is based on “Thoughts in Flight: Automation Use and Pilots’ Task-Related and Task-Unrelated Thought,” by Stephen M. Casner and Jonathan W. Schooler, published in Human Factors: The Journal of the Human Factors and Ergonomics Society, Volume 56 (May 2014): 433–442.