Intentionally Noncompliant

Flight crews that are intentionally noncompliant with cockpit standard operating procedures are two to three times more likely to commit other, unintentional errors or to mismanage threats to flight safety, according to data gathered during thousands of line operations safety audit (LOSA) observations.

Intentional noncompliance errors are common, James Klinect, chief executive officer of The LOSA Collaborative, said in a presentation to Flight Safety Foundation’s 66th International Air Safety Summit (IASS), which was held in Washington in late October.

Klinect’s organization has collected more than 20,000 LOSA observations at more than 70 airlines worldwide, beginning in 1996 (Figure 1). LOSA observations are governed by 10 operating characteristics, including jump-seat observations by trained observers during regular operations; voluntary crew participation; anonymous, confidential and nonpunitive data collection; and feedback to line pilots (see “LOSA Operating Characteristics”).

Those observations show that, as the number of instances of intentional noncompliance increase on a flight, the number of mismanaged threats and errors and the number of instances of an undesired aircraft state also increase, Klinect said (Table 1).

All flights in the LOSA archive, regardless of the number of intentional noncompliance errors, had about the same average number of threats, according to LOSA data. However, in flights without intentional noncompliance errors, an average of 2.1 unintentional errors occurred per flight, compared with averages of 3.9 unintentional errors in flights with one intentional noncompliance error and 7.5 unintentional errors in flights with two or more intentional noncompliance errors.

Klinect told the IASS that on 49 percent of the flights in his organization’s LOSA archive, the observer reported at least one intentional compliance error.

The real problem, he added, is that flight crews often respond incorrectly to an intentional noncompliance error — that is, their response actually represents a mismanagement of the error — about 20 percent of the time, “so 20 percent of intentional noncompliance errors are actually linked to other errors.”

The most common of the intentional noncompliance errors include omitted altitude callouts, checklists performed from memory, failure to execute a mandatory missed approach, a pilot making flight guidance changes while hand flying, and taxi duties that are performed while an airplane is still on the runway.

“Very simple things,” Klinect said. “Don’t these things seem really minor? … I would argue that … until we start reading what the intentional noncompliance is. … It’s a big deal when you take it from a system view … and [see] what a culture of noncompliance can create.”

As an example, he cited a LOSA observer’s description of the following intentional noncompliance error involving a flight crew on a Boeing 777-300ER.

With the airplane at 2,800 ft and established on an instrument landing system approach, air traffic control (ATC) told the crew to maintain 160 kt. Instead, the captain told the first officer (FO), who was the pilot flying, to “ignore it so that the aircraft could be stable by 1,500 [ft],” the description said. The mode control panel was set for 126 kt, and the crew selected 30 degrees of flaps. The FO “offered” to set the airspeed at 160 kt, but the captain “was adamant that [the ATC instruction] was to be ignored,” the description said, adding that at no time did the captain tell ATC his intentions.

The LOSA observer classified the error as “intentional speed deviation without ATC clearance/speed too low.”

Procedural Drift

Klinect said that occurrences of intentional noncompliance are a way of measuring an organization’s procedural drift, which he defined as, “if you have a set of procedures that are written … and flight crews drift away from how things are written.”

One question that should be asked at every airline, he said, is “how far do you let your guys drift?”

LOSA observers evaluate the extent of procedural drift by observing flight crew actions and listening to the comments they make in the cockpit, Klinect said.

As an example, he cited the instance of a flight crew that commits the same error several times, such as missing a level-off call four or five times during one phase of flight. Together, those missed calls are coded by LOSA observers as one intentional noncompliance error.

“We can’t get in the guy’s head to ask him whether it was intentional or unintentional,” Klinect said. “We try to use observables. … It’s not perfect, but it gets us close.”

In other instances, the crew “openly discusses that they’re going to break SOPs [standard operating procedures], and, yes, this happens in front of observers,” he said. Often in these cases, the pilots believe that they are saving time — “performing a checklist from memory is a classic,” he said.

Managing Errors

Analyses of LOSA observations have found that the best flight crews are those that not only manage the operational complexity of their flights but also anticipate threats and errors, and manage those, too, Klinect said. They use threat and error management (TEM) defenses that include policies and procedures, monitoring/cross-checking, crew resource management, checklists, deviation callouts, aircraft hardware, airmanship and “luck.”

Weaknesses in TEM defenses become obvious during LOSA observations, Klinect said, noting, as an example, that checklist errors occurred during 26 percent of LOSA archive flights — most of them during predeparture.

He also noted that flight crews who are rated poor/marginal in their monitoring and cross-checking skills are two times more likely during LOSA flights to experience mismanaged errors and undesired aircraft states.

“That’s why we focus on monitor/cross-check,” Klinect said. “It’s not monitoring for the sake of monitoring. … Threat and error management tries to bring … the things that we want to focus on — managing threats, errors and undesired aircraft states.”

Nearly all of the 20,000 flight observations include some type of crew error, and “unfortunately, not all flight crews manage errors,” he said. “In fact, about 35 percent of the flights that we see get into aircraft deviations, unstable approaches, speed deviations [defined by LOSA as plus or minus 10 kt from an ATC speed clearance or ATC speed restriction], vertical deviations or lateral deviations.”

EBT Initiative

LOSA data support evidence-based training (EBT), pointing to areas in which pilots would benefit from further training, Klinect said.

The primary pilot mismanagement issue involves speed deviations while hand flying an aircraft, he said, noting that data show that 81 percent of these occurrences are mismanaged. Most occur during descent/approach, and the problem is rarely called out by the pilot monitoring, he said, adding that “the procedures are there, but the guys don’t do it, for some reason.”

He said that at least two areas where data indicate a need for additional training involve crew interactions with ATC — coping with ATC radio transmissions that include three or more instructions, which are mismanaged about 19 percent of the time, and coping with “challenging” ATC speed clearances, which are mismanaged about 13 percent of the time. Overall, 61 percent of the archived observations involve some sort of threat related to ATC, he said.

Other frequent threats are thunderstorms, which are mismanaged about 12 percent of the time — typically because of issues involving weather radar usage or an uncertainty about when and how far to deviate around the storm.

In other areas, pop-up aircraft malfunctions are mismanaged 16 percent of the time, often during predeparture, with crews using shortcuts, such as pulling a circuit breaker, to save time, Klinect said. Data also show 46 percent mismanagement of the detection of automation errors, and 35 percent mismanagement of flight management system entry errors during predeparture.

Unstable Approaches

Four percent of these LOSA flights had an unstable approach, Klinect said, noting the LOSA definition of a “stable approach” as one in which — during the last 1,000 ft above airport elevation in instrument meteorological conditions and the last 500 ft in visual meteorological conditions — the airspeed is between 5 kt below target speed and 10 kt above, with a sink rate no greater than 1,000 fpm, and within one dot of the glideslope and localizer centerlines.

In 87 percent of these flights, the crews continued to an uneventful landing; in 3 percent, the crews conducted a missed approach; and in 10 percent, the airplane continued to the runway and was landed long, short or “significantly off [runway] centerline,” he said.

Of the unstable approaches among flights in the LOSA archive, 30 percent were associated with identified threats such as challenging clearances from ATC or weather problems. The remaining 70 percent had no clear association with a threat but were related instead to poor hand flying skills, Klinect said.  He noted that pilots monitoring were more likely to speak up about the unstable approach in cases involving a threat than they were in cases involving poor hand flying.

The LOSA observations also revealed a number of airlines with unstable approach rates of less than 1 percent, he said. Those airlines typically have implemented best practices that call for aircraft to meet stabilized approach criteria 1,500 ft above airport elevation — to give the flight crew more time to ensure that the airplane is at the correct speed well before landing. Those airlines’ best practices also call for conversations with pilots who have flown unstable approaches to discover the circumstances surrounding the events.

Using LOSA Data

At many airlines, LOSA data are turned over to flight operations, flight standards or training personnel who already are overloaded with other responsibilities and “always fighting the latest crisis,” Klinect said.

As a result, they have little time to devote to proactively addressing the issues derived from analysis of safety data.

“There’s no crisis or urgency associated with predictive data, and that’s a problem,” he said, recommending that airlines establish an independent LOSA review board, made up primarily of line pilots, to review LOSA data to identify problems and present solutions to airline management.

Finally, LOSA-inspired safety actions must have the support of management, he said, adding, “It sounds so cliché, so Business Management 101. But it’s true.”

Update

James Klinect, chief executive officer of The LOSA Collaborative, said, in a clarification after publication of an article about line operations safety audits (LOSA) in the December 2013–January 2014 issue (“Intentionally Noncompliant”), “It’s not really how a flight crew responds to intentional noncompliance (INC) errors that dictates INC mismanagement. It’s the outcome, regardless of how a crew responds. … In LOSA we call bad outcomes, regardless of response, mismanagement.”