Making a List

In the early days of aviation, it became clear that flight operations — particularly cockpit procedures — needed to be standardized and documented to enhance safety. So the industry pioneered the checklist, a standardized work-instruction document for aircraft flight crews and maintenance technicians to perform tasks in accordance with a clearly defined sequence of steps designed to ensure that necessary tasks are not skipped or forgotten. Considerable gains in safety have been achieved by the standardization of procedures in flight operations and the monitoring of other line personnel that is enabled by the use of a checklist system.

Checklists now are being used in other aviation domains where standardization is desirable, such as ground handling, and in other industries and disciplines, such as medicine, that are characterized by heavy workloads, a small tolerance for error and the handling of sophisticated equipment. Also, new trends in checklist development have emerged, including the transition to paperless checklists enabled by the adoption of electronic flight bags, the use by pilots of tablet computers, and advances in electronic flight instrument systems with integrated checklists.

But not all checklists are created equal. Errors in the design and use of checklists have been the object of attention of aviation human factors practitioners since the 1980s. The complexity of flight operations, the often-heavy crew workloads, and the limitations of human performance remain significant issues affecting checklist design and use.

Experience has shown that the development of checklists needs to be guided by common principles. European implementing rules on air operations, for example, require operators to establish for each aircraft type a checklist system to be used by crewmembers — in all phases of flight under normal, abnormal and emergency conditions — to ensure that the procedures in the operations manual are followed. The implementing rules also require that the design and utilization of checklists be based on human factors principles and take into account the latest relevant documentation from the aircraft manufacturer.

Standardization also is one of the primary objectives of checklist utilization. According to scientists Asaf Degani and Earl L. Wiener, it is sound management “to set a goal of minimizing the differences in operational procedures between fleets in order to aid pilots in transitioning from one aircraft type to another.”¹ However, they said, “overemphasis on this issue can hamper the effectiveness of the checklist. This is apparent when a checklist sequence in the cockpit does not correspond to the location of items, or when items which are not significant in one aircraft are added for the sake of standardization with other aircraft.”

Standardization in checklist design also is associated with compliance with regulations and company practices. However, Barbara Burian, a research psychologist at the U.S. National Aeronautics and Space Administration (NASA) Ames Research Center, said, “Crews should be reminded in checklists that SOPs [standard operating procedures] and regulations can and should be violated to the extent necessary if the safety of the aircraft and crew warrants doing so in an emergency.” As an example, Burian cites a 1996 emergency on a Douglas DC-10 in which the first officer began to slow to an airspeed of 250 kt to comply with regulations requiring 250 kt or less below 10,000 ft, but was urged by the captain to “keep the speed up” because of an uncontrollable cargo fire on board and the urgent need to perform an emergency descent and landing.²

“Many emergency and abnormal checklists are written with separate sets of steps to be completed depending upon the specifics of the situation being faced. Likewise, it is not uncommon for crews to be directed in one checklist to additional checklists or other information, such as aircraft performance tables, when responding to a single malfunction,” says Burian. The International Federation of Air Line Pilots’ Associations (IFALPA) recommends that “in an abnormal/emergency situation both crewmembers should be able to select and use the appropriate checklist, without diverting a large amount of attention locating that checklist.” IFALPA also recommends that familiarity with abnormal/emergency checklists should be trained in the simulator.³

Checklist Length

When designing a checklist, determining the appropriate length and level of detail is a significant concern.

“Some will argue that most of the configuration items required in operating the aircraft must be presented in the checklist. Others will argue that since the checklist is a redundant task, only the critical and most important items should be presented on the checklist. This statement leads to another controversy — which items are critical and important enough to be registered on the checklist,” say Degani and Wiener.⁴

Burian says, “Cognitive limitations experienced by humans when dealing with stress, concurrent task demands and time pressure (e.g., decreased working memory capacity) underlie many of the errors made by crews when responding to emergencies. Including more information in checklists can reduce memory load and other cognitive demands. However, the more information is included in a checklist, the longer it becomes and the more time is needed to complete it.”⁵

The appropriate level of detail and the length of a checklist should be evaluated on a case-by-case basis, she says.

Typography

Degani, a scientist in the Intelligent Systems Division at NASA Ames and the author of On the Typography of Flight-Deck Documentation, a research report, provides recommendations for the checklist developer to follow with regard to typography.

Degani says that when other typographical factors are controlled, roman fonts are less legible than sans-serif fonts — more contemporary fonts that do not include the little strokes (serifs) that project horizontally from the top or bottom of a main stroke. Most human factors–based handbooks of documentation design advocate the use of sans-serif fonts, emphasizing the desirable characteristics of modern sans-serif fonts as being clean and simple. It is also true, however, that serifs at the top and bottom of a character create a “railroad track” for the eye to follow along each line of printed text. Therefore, adequate spacing between lines of text should be used in order to prevent the eye from bridging (slipping) to the adjoining line. The checklist designer should safeguard against this factor as it may lead to skipping a line while reading a long list.⁶

With regard to font height, Degani says that a font size between 0.14 and 0.20 in (3.6 and 5.1 mm) is suitable for checklists and other critical documentation used on the flight deck. However, for practical reasons of limited space on a single page (in the case of a checklist) and simply because of increased size/weight (in the case of printed manuals), this range may not be efficient if the document is either accessed in a paper format or if its display on tablet computers requires considerable effort in zooming and scrolling.

Burian says that “typically, electronic displays allow for far fewer lines of text to be shown on a screen than can be shown on a single page of a paper checklist. Thus, even short paper checklists become multiple ‘page’ electronic checklists, and designers must decide the best ways for crews to access all of the items within a single checklist, such as through scrolling or paging conventions.”⁷

How far can the designer go in reducing the font size of checklists? The answer is based on the specifics of each case. What is clear is that a font size below 0.10 in (2.5 mm) for any important flight-deck document is not recommended, says Degani.⁸

Another recommendation by Degani concerns the slope and angular alignment for reading. An operator should make clear that the company standard for checklist reading is the optimum reading condition achieved when the printed copy — or the digital display — is held flat and the plane of the document/screen is at 90 degrees to the line-of-sight.

Phraseology

Degani and Wiener list a set of checklist-–applicable design guidelines available in the human factors literature: restricting vocabulary size, using phonetically balanced words, increasing sequential constraints between items and employing frequently used words.⁹

IFALPA supports the use of plain language in the creation of checklists. “It should be noted that a majority of users [do not have] English [as a] mother-tongue, and effort should be made to avoid confusing or ambiguous wording,” reports IFALPA.¹⁰

Some checklists may be written with ambiguous responses such as “set,” “check” or “completed” to indicate that an item has been accomplished. Degani and Wiener believe that, whenever possible, the response should convey the actual status or the value of the item (switches, levers, lights, fuel quantities, etc.). They cite as an example calling out specific “V speeds” while checking the airspeed bugs prior to takeoff and landing. By calling out the actual numbers (and V₁, V_r,, V₂, V_REF, etc.) as a response, the pilots have a verbal confirmation of the setting they have placed on their respective airspeed indicators. In addition, this overt recall can aid in the mental preparation for takeoff/landing.¹¹

Ordering Checklist Items

The limited capacity of short-term memory is one of the greatest constraints on human performance, according to Degani and Wiener. Yet many tasks, such as reading long checklists, place unrealistic demands on this memory. Nevertheless, certain design techniques aid human abilities under these limitations. The users can chunk (group) information into defined units regardless of the length or size of the unit, for example. The chunk is created when two or more items share a common factor that aids in “gluing” these items together in memory.¹²

Chunks can be employed in designing the structure of a long task-checklist by grouping the items corresponding to a system such as pressurization, hydraulic, electrical, etc., to chunks of checklist items and by graphically separating these chunks while designing the layout of the checklist card. Following this design principle, it will be easier for the pilot to index and follow the order of the items while conducting the checklist, as the sequence of movement will be within and between chunks.

If the chunking technique helps overcome the limitations of short-term memory, additional factors should be considered for the safe and effective ordering of checklist items. Carl Norgren, formerly an airline captain and currently a flight operations safety specialist, underlines the importance of human performance considerations (some of which already affect cockpit design) for the safe design of checklists. He says that “the typical working environment of a pilot is characterized by a large variety of ‘buttons,’ which, depending on the phase of flight, need to be in a specific position and activated in a specific sequence, which needs to follow a ‘natural’ flow according to the cockpit physical realities and based on visual perceptions and on other ergonomic considerations such as creating a smooth flow of movements.”

According to IFALPA, “layout and flow should correspond to the ‘geography’ of the cockpit to simplify the execution of checklist actions and reduce the risk of omissions or incorrect switching.”¹³

With regard to the ordering of checklist items, Norgren recommends that critical tasks be performed early by design of the checklist; in fact, he says that the probability of forgetting to perform a task increases as the pilot proceeds with the checklist. “When ordering checklist items, the designer should avoid the duplication of items, except of course for exceptional ‘killer items’ such as flap settings, which are advisable to double check. Every checklist should have a defined end-point and a positive confirmation that the checklist has been completed,” says Norgren.

Decoupling

Borrowing Charles Perrow’s concept of coupling — a term in software engineering and other system design that relates to the amount of slack or buffering between system components (a factor that is usually time-dependent) — Degani and Wiener say that within the reality of aviation line operations, checklists are “tightly coupled systems.” The crew can couple the checklist procedure to other tasks, such as starting engines, takeoff, landing, etc. A checklist can be coupled by design, such as a takeoff checklist that must be accomplished on the active runway, or just prior to entry onto the runway. When a checklist is tightly coupled, the buffers embedded in the system (redundancies and backups) are bypassed, and the ability of the crew to recover from a failure is diminished.¹⁴

It is advisable for system designers to decouple checklists, some specialists say. With regard to takeoff checklists, Degani and Wiener say that “the critical checks should be completed earlier in the ground phase in order to decouple the critical items from the takeoff segment as well as to allow enough time (buffers) for the crew to detect and recover from a configuration failure.”

Crew Resource Management

“Properly designed checklists can be a very valuable tool to support crew resource management. They promote mutual monitoring, allow mutual support during periods of high workload, lead to an optimum division of labor and enable the coordination among crewmembers. In a sense, checklists are the core of teamwork,” says Norgren.

According to researchers Loukia Loukopoulos, Key Dismukes and Imma-nuel Barshi, “The concept of crew monitoring [ASW, 11/14, and <flightsafety.org/toolkits-resources/flight-path-monitoring/>] in aviation holds that each pilot has a formal responsibility, in addition to all other duties, to cross-check crucial actions of the other pilot and to periodically assess the flight path of the aircraft and the status of its systems.” They also say that, “as with checklists, however, monitoring is also vulnerable to lapses. Organisations can mitigate these lapses by formally describing in their operating manuals what is to be monitored and how, and by treating monitoring as an essential task rather than as secondary.”¹⁵

Timing of the initiation of any task-checklist should be carefully considered by the captain. Degani and Wiener say, “The captain must make the following cockpit management considerations: Are other pilot(s) overloaded with other tasks? What are the consequences of having the other pilot(s) running the checklist and, therefore, not participating in the current task? What is the likely outcome of delaying the checklist because of the above considerations?”¹⁶

“Following the initiation of the checklist, the captain must also constantly evaluate the quality of the checks performed by himself and other pilot(s). If due to any factor (i.e., interruptions, distraction, time limitation, workload), the quality of checklist performance appears to be below the acceptable level, it is the captain’s responsibility to stop the checklist, allocate additional time for proper execution, and possibly run the checklist again,” they said.

Error Management

“The reality of line operations is such that there are many stress factors affecting crew performance: Time pressure, fatigue, interruptions, bad weather, traffic density at airports and delays all take humans to the limits of their performance and in such circumstances errors can occur,” says Norgren.

A typical error, according to Norgren, occurs when a checklist is interrupted. Because of the limitations of short-term memory previously noted, the wrong item may be put on hold, and this may lead to checklist items being skipped. “Traditionally, on paper checklists, pilots have used their thumbs as indicators of hold items. In this respect, the transition to electronic checklists is of help as they allow for a more accurate definition of the progress made in checklist accomplishment. There exist software applications accessible from tablet computers that allow checklist items either to disappear from the list and leave room for the following items or to change color once they have been performed,” says Norgren.

To summarize, checklists are at the core of aviation operational standardization, but lapses in their development and accomplishment can lead to undesirable outcomes. The management of checklist development and implementation needs to account for both the peculiarities of the equipment operated and the fallibility of the humans in the loop. 

Notes

1. Degani, Asaf; Wiener, Earl L. Human Factors of Fight-Deck Checklists: The Normal Checklist. Contract No. NCC2-377. A report prepared for Ames Research Center. May 1990.
2. Burian, Barbara. “Design Guidance for Emergency and Abnormal Checklists in Aviation.” In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. 2006.
3. IFALPA. Flight Documentation, Position Paper 15POS05. July 22, 2014.
4. Degani; Wiener.
5. Burian.
6. Degani, Asaf. On the Typography of Flight-Deck Documentation. Contract No. NCC2-327. A report prepared for NASA. December 1992.
7. Burian.
8. Degani.
9. Degani; Wiener.
10. IFALPA.
11. Degani; Wiener.
12. Degani; Wiener.
13. IFALPA.
14. Degani; Wiener.
15. Loukopoulos, Loukia D.; Dismukes, R. Key; Barshi, Immanuel. The Multitasking Myth – Handling Complexity in Real-World Operations. Burlington, Vermont, U.S.: Ashgate, 2009.
16. Degani; Wiener.

---

Featured image: © Serge Bailleul | AirTeamImages