Reporting Climate

For a fatigue risk management system (FRMS) to work effectively, the host airline must develop a strong safety reporting climate, according to a report published by the International Civil Aviation Organization (ICAO).¹ In a paper describing their own research, Michelle Harper and Robert Helmreich explain “reporting climate” as “a component of safety climate that is characterized by the beliefs and attitudes that operators hold towards the reporting of their own errors and the behaviors that characterize the use of reporting systems.”²

The ICAO report says, “Both SMS [safety management systems] and FRMS rely on the concept of an ‘effective safety reporting climate,’ where personnel have been trained, and are constantly encouraged, to report hazards whenever observed in the operating environment.” This advice is repeated throughout the document.

The consequences of operating an FRMS in the context of a deficient or nonexistent safety reporting climate are potentially catastrophic. An FRMS is an example of a single-loop control system. In a single-loop control system, data (feedback) are used to regulate and optimize. A heater-thermostat combination is an example of a single-loop control system: By monitoring the physical environment, the thermostat generates control inputs. Acting as the interface between the heater and its physical environment, the thermostat ensures that the system operates within an acceptable, pre-programmed, range.

As with home heating, so too with rostering: Fatigue and incident reports, debriefings, sleep logs, Samn-Perelli scores,³ reaction time tests and actigraphy traces⁴ generate the data required to validate and optimize flight crew rosters, such that a balance is maintained between resource utilization, or economic performance, and fatigue, or safety performance. By mediating the relationship between rostering and the operating environment, fatigue data analysis — the cornerstone of an FRMS — ensures the system operates within acceptable parameters, synthesized from national and international regulations, productivity agreements and the scientific literature.

As mentioned above, problems arise when feedback is stymied. In 2010–2011, the author investigated the pilot lifestyle.⁵ The research, funded by the British Air Line Pilots’ Association, generated interview data, 433 questionnaire returns and more than 130 sleep logs (SLOGs), most of which ran to several thousand words (see, “The Pilot Diaspora”). The data showed that some pilots preferred to report sick rather than admit to being fatigued.

Typical comments were these:

• “We tend to position home after early shifts, making our days 12 hours long on average. I have called in sick numerous times, simply because I felt the company’s response to ‘I am fatigued’ would be harmful to my career.”
• “My fatigue report has been rejected. The only way to do it is to go sick. It saves a call from the management.”
• “On these runs of five earlies, particularly on days four and five, you are absolutely buggered. But because it happens all the time, you get used to the fact that that is how you feel. So you turn up anyway. If you don’t, you get snotty emails. You get pulled into the office. People get disciplined for being off sick. They have even started saying that people use ‘fatigue’ far too often. So one of the few things we can say without being questioned is now being questioned, because they think we say it too often.”
• “You can work to the rules and still be very, very fatigued. Airlines are now working you right up to the very limit of the rules. My airline is quite good. If I say I am fatigued, they won’t question it. Not all airlines are like that. They say if it’s legal, you have to come to work, despite being fatigued. There was no system for reporting fatigue at my previous airline. It was either report sick, or go to work.”
• “I had two days of the simulator with a CAA [U.K. Civil Aviation Authority] inspector in the right seat who told me I looked absolutely exhausted, and that I should go sick for my next trip! It took someone that senior to tell me to do it or I might still have gone to work (stupid, I know!).”
• “[I have refused a duty] only once, when due to do a four-sector day starting at 0615. I had trouble sleeping and was still wide awake at 0300. It was recorded by the company as sickness.”

A reluctance to report fatigue is not peculiar to Europe-based pilots, as noted in this report by the U.S. National Transportation Safety Board (NTSB):

• “Some of the air carrier pilots reported using [fatigue risk management programs] successfully, whereas other pilots reported that they hesitated to use such programs because of fear of retribution. … In addition, other pilots reported that they attempted to call in as fatigued but encountered company resistance.”⁶

Without data, rosters cannot be certified as safe. The possibility then arises that unsafe rosters permeate airline operations. Unsafe rosters may be thought of as latent errors or resident pathogens — “bugs,” if you like. Under certain conditions, such bugs may cause or contribute to incidents or accidents: Latency becomes active. The problem is compounded by the fact that it is impossible to predict the conditions for activation. Given these facts, it is best to minimize the possibility of unsafe rosters at the outset … by ensuring that fatigue is unfailingly and accurately reported. To invoke the heater-thermostat metaphor, one must ensure that the thermostat is fully functional and correctly calibrated at all times.

On the face of it there is a simple answer to the problem of non-reporting or masking — reporting climate surveys. Harper and Helmreich argue that a reporting climate is influenced by five factors:⁷

• Perceptions of agency — “The degree to which a person sees a reporting system as a viable place to create change will be a strong determinant of the organization’s reporting rate.”
• Protections — “Those reporting systems that offer higher levels of protection [from disciplinary action and litigation] will benefit from higher reporting rates.”
• Employee confidence in management’s commitment to safety — The greater the employees’ confidence in management’s commitment to safety, the higher the reporting rate.
• Ease of use —The more user-friendly the reporting system(s), the higher the reporting rate.
• Notions of personal responsibility — “Operators [in this case, pilots] with stronger opinions of personal responsibility will be more likely to use a reporting system.”

A reporting climate questionnaire should evaluate at least the five dimensions listed above. The questionnaire should quantify masking and non-reporting. Explanations should also be sought from pilots via questionnaire and/or interview. The questionnaire should be anonymous and issued to all pilots, including management pilots. Approval of an airline’s FRMS should be conditional upon the successful completion of regular reporting climate surveys, and, if required, of remedial action.

Reporting climate surveys are problematic. Survey response may be influenced by several factors, including the perceived credibility of those conducting the survey. In recent years, there has been a move toward “light touch” regulation, with those regulated bearing a greater responsibility for performance monitoring and remedial strategies.⁸ While self-assessment is attractive on financial grounds, because the logistical burden is passed to those regulated, self-assessments may be considered less credible than assessments conducted by third parties. Even if an airline’s assessment is demonstrably objective, it is still possible the assessment will be viewed as biased in favor of commercial interests. Perceptions of bias may reduce participation levels to the point where the survey lacks credibility. To eliminate this risk, the reporting climate survey must be administered by a disinterested third party.

To summarize: Fatigue risk management systems work only if there is a sufficiency of data. For there to be a sufficiency of data, pilots must unfailingly report fatigue episodes. Non-­reporting or masking undermine fatigue risk management systems because the data required to validate rosters are lost. Non-validated rosters represent latent errors or resident pathogens within flight operations. When unpropitious circumstances accumulate to the point where system defenses are breached, such errors may become active (“live”) with possibly catastrophic results. Given the potential consequences of non-reporting or masking, the approval or re-approval of an airline’s FRMS must include a reporting climate survey administered by a disinterested third party. If the regulator is perceived to be too close to the airline’s management, the survey must be administered by another party — for example, by a consultancy or university familiar with reporting climate surveys.  

Simon Bennett, director of the University of Leicester’s Civil Safety and Security Unit, has a doctorate in the sociology of scientific knowledge. He has been a consultant to the airline industry for more than a decade.

Notes

1. ICAO, International Air Transport Association and International Federation of Air Line Pilots’ Associations. Fatigue Risk Management Systems — Implementation Guide for Operators. Montreal: ICAO, 2011.
2. Harper, M.L.; Helmreich, R.L. “Creating and Maintaining a Reporting Climate.” In Proceedings of the 12th International Symposium on Aviation Psychology. Dayton, Ohio: Ohio State University, 2003.
3. The subjective Samn-Perelli checklist measures fatigue on a seven-point scale.
4. An actigraph is a small device worn on the wrist that contains an accelerometer to measure movement and a memory chip to store “activity counts” at regular intervals such as every minute.
5. Bennett, S.A. The Pilot Lifestyle: A Sociological Study of the Commercial Pilot’s Work and Home Life. Leicester, England: Vaughan College, University of Leicester, 2011.
6. U.S. National Transportation Safety Board. Runway Overrun During Landing, Shuttle America Inc., Doing Business as Delta Connection Flight 6448, Embraer ERJ-170, N862RW, Cleveland, Ohio, February 18, 2007. Washington, D.C.: NTSB.
7. Harper and Helmreich, op. cit.
8. According to the Safety and Health Practitioner, the U.K. government’s Transforming Regulatory Enforcement program involves “a review of all regulators … to make sure each one is making the fullest possible use of alternatives to conventional enforcement methods.” Safety and Health Practitioner, December 7, 2011. www.shponline.co.uk.