Recent reports to the U.S. National Aeronautics and Space Administration (NASA) Aviation Safety Reporting System (ASRS) show an increase in checklist errors. Flights roll onto the runway for departure only for the pilots to discover that flaps are not properly set for takeoff. A flight takes off and the autobrakes are not set for a rejected takeoff. A flight departs and climbs out only to discover that pressurization packs are not on.
How can that be? Setting the flaps for takeoff is on the checklist! Making sure the autobrakes are set is on the checklist! Checking that the packs are on and pressurization is set is on the checklist!
What’s going on here? Is something wrong with our pilots? Have we lost our checklist discipline? Or is something wrong with our checklists?
As is often the case, the truth is somewhere in between. That is, the problem is in the interaction. Our procedures and checklists are not always sensitive to our humanness, and we encounter conflicts between procedural demands and the demands of the operation or of being human. By design, the checklist is full of traps. We run checklists so often and so well that it’s easy to forget those traps. But once we forget them, we are much more likely to fall into them. Just the fact that we run the checklist so often is a trap in itself. The pattern becomes automatic. We go through the words and don’t go through the motions, like the pilot who responded to the call for the flaps with “set” when the flaps weren’t set or they were set to the wrong setting for the given takeoff. It’s a ritual. It’s that “sing song” we perform every time, but we are tuned to the melody and don’t really pay attention to the words, even though we mouth them. Or we hear the words, but through repetition, they have become dissociated from their meaning.
Expertise is marked by automaticity. As experts, we do things quickly and smoothly, without effort and without having to allocate cognitive resources. For example, after years of driving a standard transmission, we shift gears without thinking. Psychologists have been making a distinction between automatic and controlled cognitive processing, or as Daniel Kahneman recently dubbed them in his book Thinking Fast and Slow, System 1 and System 2:
- System 1, the automatic processing, is fast and effortless.
- System 2, the controlled processing, is slow and effortful.
Being cognitively efficient, we rely on System 1, and only when we get stuck do we shift to System 2. Thus, we are caught in a paradoxical situation. On the one hand, we want to be expert pilots who do things quickly and smoothly with little effort. And at the same time, we are called to execute the checklist under System 2’s controlled processing, paying close attention to everything.
If I were to sit on your jump seat while you hand fly and if I tapped your shoulder asking, “Why did you apply left aileron just now?” you would probably be surprised to hear the question, shrug your shoulders and say, “Did I apply left aileron? It probably needed it.” And you’d be perfectly right. But if I were to tell you that you said “set” without looking at the flaps handle or the flaps gauge, you’d be upset. At NASA, we have done studies using eye-tracking to see where pilots look while flying and found that we don’t always look. We sometimes look and don’t see, or see what we expect to see rather than what’s actually there. Expectation bias is a powerful mechanism. Like most biases, it usually works in our favor, but every so often it trips us. And the checklist trips us, too.
Any checklist with two or more line items is a trap, an invitation for an interruption. And what happens when we are interrupted in our flow? We are hard-wired to notice change. And we are hard-wired to respond to interruptions, especially of the verbal kind. As social animals, when we hear speech, we turn our heads toward it.
When someone calls our name, even in a noisy crowded room, our ears perk up. Studies of interruptions in the cockpit show that a large majority are triggered by speech and communication. And when we are interrupted, we stop whatever it is we are doing, even if only briefly, to evaluate the interruption: Is this interruption something I need to pay attention to?
When we are interrupted, we suspend whatever we were doing and divert our attention to the interruption to make a quick judgement: Do I defer the interruption, or do I defer the task I was interrupted from? We can’t pay attention to both things at the same time. There is no such thing as multitasking when the tasks require attention. Let’s interrupt ourselves here for a moment (and remember to get back to where we were; highlight the place in the text and add a comment in the margin as a reminder) to expand on this point because there is so much talk these days about multitasking and the perceived need to multitask.
The human attention mechanism is a “serial processor,” not a “parallel processor.” All of us can breathe, circulate blood through our body and digest food at the same time but only because we don’t need to pay attention to any of it.
When we must pay attention to more than one thing at the same time, we engage in what psychologists call “task switching” – going back and forth between the tasks. But switching has a cost. Every so often, for example, something falls in between, or we forget where we left off and switch back to the wrong place like returning to the checklist a line or two past where we left it when we were interrupted. That’s why some people argue that any time you are interrupted in the execution of a checklist, you should start from the beginning and do it all again. But we don’t always have the time to do that, and besides, we feel confident that we know exactly where we were before and what items we already accomplished.
That confidence might be the result of what psychologists call “memory source confusion.” Especially when we fly multiple legs, we sometimes can’t tell if we did one specific line item just now, or on a previous leg. If on top of that, we are a bit tired because the baby didn’t sleep well last night or people had a party next door in the layover hotel or it has just been a long day, these memories of the different flights can mesh together and we can no longer tell on which leg we did what.
But let’s get back to where we were before we were interrupted. Yes, writing, like attentional thinking, is serial. There was no way for me to tell you at the same time about deferring tasks and about multitasking or memory source confusion. We can only write, and we can only pay attention to, one thing at a time. And so when we are interrupted, we must suspend what we are doing and judge the interruption. And if we can’t just ignore the interruption, we must decide which task to defer: the interrupted task or the interrupting task. By the way, even if we choose to ignore the interruption, the suspended original task is already deferred in some sense. And when we defer a task, we must form an intention to return to that task later. We must form what psychologists call a “prospective memory cue.”
Unlike retrospective memory, which is when we remember things past, prospective memory is our intention to remember something in the future, like when we want to remember to send a birthday card to a friend or to return to the checklist after responding to an ATC call. But we need a cue to remind us of that intention, because prospective memory is fragile and vulnerable to interruptions. So we highlight the place in the text where we interrupted ourselves, and we create birthday reminders on our cellphone calendar. And we keep the checklist card in our hand with our finger on the last item we did, stick it between the throttles, or bring the checklist up on the multipurpose control display unit or electronic flight bag while responding to air traffic control (ATC), so we can be sure to return to it later. But then if the pilot flying calls for some action or ATC has more to say, we may fall into a trap because our checklist has two or more line items.
Closed-loop items on electronic checklists (ECLs) are great at showing us what’s been done already and what is still to be done. But not all aircraft have ECLs and not all items are closed loop. And sometime the flaps are set for a takeoff, just for a different takeoff than the one we are doing. When Boeing 777s rolled onto the departure runway with flaps retracted, it was sometimes found that the checklist on the screen was the “After Start” checklist. Something interrupted the crew from calling up the “Before Takeoff” checklist on the ECL. So on the 787, the ECL was hooked up to the GPS and when an aircraft senses that it’s close to the departure end of the flight management system‒selected runway – the “Before Takeoff” checklist comes up automatically. But as they say, if you make it fool proof, somebody will come around and invent a better fool.
So what can you do not to be that fool? While the engineers are trying to add more and more features to compensate for our all-too-human limitations, we must become aware of those limitations. And we must push ourselves to watch for those traps, especially when we are in vulnerable situations — when we are tired, for example, or when we fly with somebody we like and trust, or when everything seems to go really smoothly and we feel sure of ourselves. We must ask ourselves:
- “What trap is the checklist setting up for me next?”
- “What trap is my System 1 fast thinking setting up for me now?”
- “What truly powerful cue can I form to remind me of this prospective memory task that just landed in my lap?”
- “Did it really happen just now, or was it on a previous leg?”
- “What do I need to do to slow down the operation so I can have time to start the checklist from the beginning now that I was interrupted?”
As psychologist Eddie Friedman used to say, “If you don’t have answers, don’t feel too badly. But if you don’t have questions, you better check your pulse.”
Immanuel Barshi, Ph.D., is a research psychologist and a senior principal investigator in the Human Systems Integration Division at the NASA Ames Research Center.
