Learning Curve

Safety culture is seen as a pillar of aviation safety and is a cornerstone in both the International Civil Aviation Organization’s and the European Aviation Safety Agency’s (EASA’s) safety management system (SMS) models. But what does this mean in practical terms? For aviation organizations, is there something credible, tangible and manageable they can work with, or is this simply meant to be an inspirational goal?

In European air traffic management (ATM), 2001 and 2002 were particularly bad years. The runway collision at Milan Linate Airport in 2001 was followed by the Überlingen midair collision the next year. Both accidents were seen as having a significant safety culture component, and so Eurocontrol took the lead in developing a safety culture approach for air navigation service providers (ANSPs). Initially it borrowed from the nuclear power sector, which had developed its own safety culture approach following the Chernobyl nuclear power plant accident in Ukraine in 1986.

After two years of research, tailoring and testing, the ATM safety culture survey approach was rolled out in 2005 and applied in four countries in different European regions. The approach slowly gathered momentum until its watershed moment — a CEO safety workshop held in Belgrade, Yugoslavia, in 2010 — when several CEOs told others that this approach had benefits and could be trusted. As of today, the safety culture approach has been applied in 33 European states, and each year there is an annual meeting at which around 20 ANSPs get together to consider common safety culture issues and problems.

These ANSPs are also subject to a periodic audit of their SMSs, whether by their regulator or via the Civil Air Navigation Services Organisation, an industry-led collective organization supporting ANSPs. One of the audit question areas relates to whether they have carried out a safety culture survey and have a safety culture improvement plan in place. In ATM, therefore, safety culture has become relatively commonplace, a part of the safety management infrastructure, and most ANSPs are thus aware of their safety culture strengths and weaknesses.

Could this apparently successful approach be applied to other aviation sectors, such as airlines, airports and airframe manufacturers?

The European Union–funded Future Sky Safety research program (2015–2019) aims to increase safety in European aviation. One project within this program focused on applying the safety culture survey approach, successfully developed and implemented in European ATM, to other sectors in aviation. Over the past four years, six independent projects (results below) were conducted. The projects included a pan-European safety culture survey of 7,200 pilots, as well as surveys in two major European airlines, two airframe manufacturers and six organizations at a single airport location (London Luton Airport).

Future Sky Safety has shown that the safety culture survey approach can indeed be grafted onto other aviation sectors, as will be demonstrated in this article. But as the Future Sky Safety program draws to a close, there are two key questions following this “safety culture experiment” that can be asked in relation to each of the surveys:

• How easily does the approach translate into other aviation sectors?
• What was the added value or benefit of the survey for the organization(s) concerned?

At the end of the article, a third question is addressed in an attempt to summarize the potential impact across all the surveys: What is the resultant business case for other aviation organizations embarking on safety culture surveys?

The answers to these questions can help inform those responsible for safety in deciding whether their aviation organizations should consider embarking on their own safety culture journey.

What follows are the results for each of the survey projects.

Airline No. 1 — easyJet

Airline No. 1 (easyJet) already had some experience with a safety culture survey but decided to expand the survey to a larger part of its operation (Figure 5-don’t need this). The survey was supported by the senior management board (including the CEO), facilitated by the company’s human factors manager, and ran well, with a healthy response rate. The workshops were also productive in terms of gaining a deeper understanding of the issues and how to improve safety culture.

The results showed certain core strengths in safety culture (Figure 1), and an associated action plan was developed and outlined at the first Future Sky Safety conference in March 2017.

In terms of the two questions raised above, persuading the organization to undertake a survey was relatively straightforward, especially as there was already a positive experience from the maintenance survey, as well as an internal champion to lead efforts to set up the survey and workshops, and to carry the process forward. The underlying structure of the survey — the safety culture “dimensions” underlying the questions, such as management commitment to safety, risk-handling and fatigue — seemed to fit well with the range of concerns of people who undertook the survey. What people said in the workshops reflected the questionnaire results, and there were not too many open (free-text) comments, which is an indicator of questionnaire element sufficiency.

The workshops were useful in developing a deeper diagnosis of certain areas — for example, fatigue as a function of flexibility in rostering, and the need for a more explicit approach to maintaining health and well-being in the organization in order to maintain operational performance.

The perceived benefits for the organization were related to the safety culture strengths confirmed by the survey, as the organization found it useful to know what it was doing well. The second main benefit was related to the required actions, which were seen as focused and manageable.

European Pilot Survey

Tailoring of this pan-European survey was supported by the European Cockpit Association (ECA), which was interested in seeing what the results would reveal about whether the new types of pilot employment contract (called atypical contracts, including pay-to-fly, zero hours, self-employed, fixed-term, etc.) had an impact on safety culture when compared to more traditional (permanent) contracts. The demographics section of the questionnaire included items concerning the type of airline (for example, network, low cost, charter, cargo or general aviation) and the type of contract.

The survey was launched with a target of 2,000 responses, but in fact attracted 7,239 valid pilot responses from 33 European airlines, spread fairly evenly across Europe (with a bias to Western European companies). No workshops were carried out for this survey due to the dispersed nature of the survey participants; however, the survey results were discussed with pilots for their interpretations. Figure 2 gives an overall picture of the results. The further out toward the edges of this figure, the better the safety culture result. This figure shows that network airlines score consistently higher than low cost and cargo airlines on most dimensions, though this does not mean that all low cost and cargo airlines have poorer safety cultures.

The biggest differences, as evident from the figure, are for perceived organizational support and fatigue management.

Comparing the overall results for pilots with an equivalent sample of European air traffic controllers suggested that there were no statistical differences between the two populations, and if anything, pilot safety culture was slightly stronger.

The report noted that pilots on unsecure contracts generally tended to score lower on safety culture than those on permanent contracts, and this led EASA to cite the report in its 2017 guidance document on managing hazards with new business models in aviation.

Perhaps most importantly, the report gave a voice to the many pilots who took part. Yet it was clear to the study team that this was not a grievance survey, since there were many positive results. Many of the open comments showed that pilots carefully considered their responses, particularly the degree to which certain factors were effectively seen as “comfort” factors, or truly affected operational safety and safety culture. For example, everyone is occasionally tired at work, but pilots’ comments made clear that they were considering whether they were tired to the extent that it could impede their performance and thus affect safety.

In terms of the first two questions raised earlier, the client approached the study team, and there was a far greater than expected return rate from the pilot community, so engagement was high and proactive. The tailoring was more demanding in this survey, to accommodate a more dispersed community, and it led to an expansion of the dimensions of the ATM version. The workshop component of the ATM methodology could not be employed in this survey, though it is still recommended for airline and other aviation organizations.

For this survey, the perceived benefits were highlighting that safety culture was generally good for pilots, except in a few areas, and it may be noted that EASA, in one of the key areas, namely pilot fatigue, is undertaking a study on the existing flight time limitation rules for how long a pilot may fly without a break.

Although an initial wave of reporting in the press, focusing on the negative survey results, was seen as a downside, this did not last, and overall, the survey had a positive effect of gaining the attention of companies and regulators alike, triggering actions in several organizations and reflection in many others. In terms of further follow-up action, it is understood that further surveys may be undertaken in the future to continue to monitor the situation for European pilots.

Manufacturer No. 1 — Airbus Design

It should be noted that for both participating manufacturers, these were seen as exploratory surveys and were focused on particular segments of the organization rather than being company-wide.

For the Airbus survey, the focus was on the Design part of the organization, which involves around 2,000 of the 70,000 employees. This survey required considerable tailoring of the questionnaire, as the job of design and systems engineering is somewhat different from airline operations, even if most underlying issues, except fatigue, and dimensions remain relevant. But the tailoring worked, with the same issues raised during workshops as via the questionnaires. Participation in the workshops was intense. The general principle espoused in each workshop was that a safety defect in a design of an aircraft could simply not be allowed to happen; therefore, any safety issues had to be resolved carefully and thoroughly before continuing, even if resolution efforts delayed progress toward production. One observation from the study team was that the managers in the management workshop were connected with safety and detailed safety concerns, more so than is often the case for middle managers.

The overall participation rate in the questionnaire was less than desired. This was probably due to the general burden of communication messages, as is often the case these days in large organizations. Those who came to the workshops stated that they wished more people had attended, as they found the workshops useful. This raises a general lesson still to be learned for safety culture — of how to overcome “survey fatigue,” since many organizations run frequent surveys throughout the year.

An issue discussed after the survey, and referred to during the workshops by several participants, was that the sensitivity to safety can vary considerably from one department or division to another. For this reason, the results from Airbus Design (which were positive), could not necessarily be ascribed to the whole organization. This has raised an unanswered question for the survey team about the degree to which organizations in different aviation segments are homogeneous in their safety culture profile, particularly when considering truly large organizations (e.g., Airbus Design, a relatively small part of Airbus, is larger than many European ANSPs).

In terms of the two questions, tailoring requirements for the safety culture questionnaire were more extensive, and possibly more tailoring would be needed if they were to be applied to the entire organization, with possible additional dimensions. The workshops were essential for Airbus and seen as the main benefit by the personnel (designers, system engineers and managers alike). No downsides were perceived. It is not known what has happened since the survey, but as stated above, this was really a first step into the realm of airframe manufacturer safety culture. One lesson linked to this aspect is the need to have an internal champion for a strong safety culture to help move it along after a survey.

Manufacturer No. 2 — Boeing Research & Technology Europe

Although most of the Boeing organization is in the United States, it has approximately 4,500 employees in 16 countries in Europe. This initial survey for Boeing Europe was led by two safety and human factors personnel at Boeing Research & Technology (BRTE), based in Madrid, supported by senior applied psychologists at the London School of Economics and focused on aircraft engineers and business support. The standard survey was adapted and applied, and interviews rather than workshops were held with a representative sample of the respondents to gather further insights into their safety culture.

As with Airbus, one lesson learned from the BRTE survey was that the ATM-derived questionnaire has a stronger focus on operational rather than production environments, and so further tailoring for manufacturers may be required.

The safety culture scores were generally positive, but one main finding from the survey in Boeing was that there were low incident/accident rates in their workplaces, and so they were not collecting as much data on precursors to incidents/accidents. As a result of the survey, actions were taken internally to enhance near-miss reporting, as well as hazard reporting and logging, involving employees in safety walks around the workplace environment, and promoting ownership of safety through employee involvement with other safety actions/initiatives. Essentially, these were seen as actions that would help to enhance what was already a reasonably good safety culture.

In terms of the two questions, therefore, as with Airbus, this was really a “taster” survey, so gaining engagement was not a big issue. Also, again (as with Airbus) it appears that for manufacturing, more tailoring might be required, as well as verification that the underlying safety culture dimensions are a good fit for the manufacturing sector. Nevertheless, the combination of interviews plus questionnaires appeared to satisfy the respondents and survey leaders that the key safety culture issues could be raised and adequately addressed.

The perceived benefits were re-confirming the areas where the company was doing things well and identifying certain areas where they could “raise their game” in safety culture. No downsides were reported, and actions were taken internally following the survey.

Airline No. 2 — KLM

The final safety culture survey of the project was carried out at KLM. This was the biggest survey of the four-year project and gained approximately 11,000 replies worldwide (Figure 3) making it the largest-ever survey for the Eurocontrol-based questionnaire, with the results analyzed by the Dutch-based NLR Safety Institute, which also led the nine workshops and key-person interviews during the survey.

At the time of writing this paper, the key results had not been released, but the overall results were positive for KLM, and there were some learning points that are currently under consideration.

The tailoring of the questionnaire took longer for this survey, due in part to the diversity of regions (e.g., ensuring that the English-based nuances in the questions would be correctly interpreted not only by Europeans but also by staff in South America, Africa and elsewhere). Additionally, there was an attempt to add some questions on security and other topics, because KLM, as is the case with many organizations, has an integrated approach to safety and security, and rather than have multiple questionnaires, it was decided to test whether the safety culture questionnaire could also address security. While the results provided useful feedback on some key security issues, it was felt by the study team afterward that security probably deserves its own questionnaire.

As in other surveys, but particularly with a large and diverse organization, an internal champion to help orchestrate the entire survey process was essential.

The workshops, as well as some one-on-one interviews with more senior management, were seen as essential to attain an understanding of the background of the survey statistics, coupling them to the way work is done in the organization, and nuancing particular statistics.

A lesson learned from this study was that there can be considerable differences among divisions of an airline in safety culture and in the way that it affects the work done. The presentation of the statistics and the organization of the workshops must account for such differences.

The survey occurred toward the end of a major reorganization, and so the company found it useful and timely to see where they were doing well, and therefore what should be maintained, and where things could be improved. In the past, certain organizations have postponed safety culture surveys due to impending or ongoing reorganizations, for fear of negativity creeping into the survey results. A lesson here was that a reorganization is not necessarily a deal-breaker for a survey. Although there may be some organizational turbulence and uncertainty during such periods, there is a greater degree of flexibility, and with people taking up new roles and positions, it may actually be easier to implement changes for the better.

The Safety Stack

One of the most innovative parts of the Future Sky Safety program was the running of simultaneous surveys at a single airport location, which led to what has become known as the Safety Stack. The idea behind the Safety Stack is simple. At an airport, there are many organizations that must work together to enable smooth and safe operations for passengers, freight and business users. Such users range from airlines, air traffic control and ground handlers to deicers, fuel services, baggage handlers, caterers and cleaning services. They are all connected. If one of them has a safety problem, then often they all do. They need to work closely together, and they already do so. Why not have them work together on safety?

The work began in late 2016 with six independent safety culture surveys of key organizations at London Luton Airport. This was the first time this had been done, and it was interesting to see the differences. Each organization had a detailed snapshot of its safety culture and its strengths and opportunities for development. It could have ended there.

But what happened next was both surprising and exceptional. The six organizations decided to share information on each other’s surveys — not the detail, but where they were doing well, and where they needed support. And they decided this should not be a one-time experience. Rather, they formed a group of (currently) 15 organizations, all based at the airport, called the Luton Safety Stack, led by London Luton Airport and assisted by Eurocontrol, which led this part of the project. It is called a “stack” because the original concept used a vertical representation of the companies, from the ground upward, and used this word to describe it.

What are the benefits? Since January 2017, when it was formed, there have been nine meetings, which usually include a workshop element. The most tangible outcome so far has been the work on ground handling. At Luton, as at many airports, there are several ground handling organizations and a number of airlines. The way operations are carried out, such as preparing for aircraft arrival, chocking an aircraft, or placing cones around an aircraft, can vary between the handlers and the airlines. This variability can lead to inconsistencies, which could allow hazards to appear, or cause delays. The Stack members have so far developed 28 harmonized procedures for all operators at Luton, for each operation creating a simple one-page procedure with diagrams, to keep it simple and safe. This initiative has already created national and international interest through the U.K. Civil Aviation Authority Ground Handling Operations Safety Team and the International Air Transport Association Ground Operations Manual (IGOM) programs.

A comparison between the first year of the Stack with the harmonized procedures, against the previous year without them, showed a 100 percent decrease in ground handling damage incidents, against a background 5 percent increase in traffic and a 7 percent rise in efficiency (on time performance).

Other Stack activities include the development of a Luton-wide safety dashboard, as well as work toward a harmonized just culture framework. The Stack partners also discuss key safety issues and detailed incidents, and how to collectively resolve them.

In terms of the two questions, it was relatively straightforward to engage the six organizations, though some tailoring was required due to the diversity of operations. The underlying dimensions appeared to satisfy the different organizations’ safety culture concerns and issues. Workshops were only used with one of the six organizations (an airline), due to resource constraints within the study team and the project as a whole. The limitation of this was that some organizations had difficulties understanding the reasons for some of the findings and how to interpret them. Many of these issues have since been discussed collectively in the subsequent Stack workshops; for example, the issue of just culture is currently at the fore, with open discussions on differences in just culture approaches in the various organizations comprising the Stack.

The benefits are clear from the above paragraphs, and the fact that the Stack has continued for almost two years and is now enshrined in the airport’s safety management system reinforces the perceived added value of the approach to the organizations involved. What is not yet clear, however, is whether this successful experiment will migrate to other airport locations. This may be about to change, as another U.K. regional airport is now discussing with Luton and Eurocontrol how to develop its own stack approach.

The Business Case

The Future Sky Safety project has shown that the safety culture survey approach, pioneered in ATM, can be translated effectively for other aviation segments, namely airlines (low cost and network), airports (including ground-handling and other diverse services) and airframe manufacturers, though more work may be required on the manufacturing side. Additionally, all the stakeholders that have participated in the surveys have found concrete benefits, with few downsides, and for most, it has become the springboard for further safety culture–related actions inside their organizations. But is this enough of an evidence base to convince other organizations to embark on safety culture surveys?

A key issue for any new organization considering embarking on such a process will be the cost. The surveys in the Future Sky project were all free for participating organizations, because the work was funded by the European Commission, but this may not be the case for future surveys. Additionally, there are the costs of internal resources, such as electing an internal champion, tailoring and running the survey and finding people for workshops.

Some organizations may also be wary of the potential results, as such surveys can be seen as a Pandora’s Box, whose outcomes may be difficult to manage, and there may even be fear of reputation damage if negative findings are leaked. In this respect, in the ATM experience of around 50 surveys, and in the Future Sky Safety surveys, no such problems have materialized. This does not mean that such fears are groundless, simply that there is no evidence to support them at present.

Such concerns need to be balanced against the benefits. It is safe to say that all organizations discussed in this paper were reassured by seeing what they were doing right in safety culture terms, and at the same time were undaunted by seeing that there were areas for improvement. In this sense, they found the process and its outcomes to be manageable and helpful.

Of course, one way to lead to more organizations engaging in the safety culture survey process is for it to be mandated — to become a regulatory requirement. This is not necessarily the best business model for safety culture, however, as it could lead to organizations conducting safety culture surveys for the wrong reasons — for example, to gain a “tick in the box.” The only reason to conduct a safety culture survey should be because the organization wants to better understand its safety culture strengths and weaknesses. Some organizations that have carried out such surveys have done so because they do not want to be surprised by some negative event or safety outcome that certain parts of the organization knew was waiting to happen. They are willing to hear bad news so as to be better informed and prepared for all foreseeable risks.

The Safety Stack has offered a fresh business model for safety culture by having an ongoing safety culture improvement process in which separate organizations with a common safety interest work together. For these organizations, the surveys were the starting blocks for a more sustainable process; indeed, only six of the 15 organizations actively working in the Stack had undertaken surveys — the others had not, though they may do so in the future. The Stack approach may have a broader appeal to aviation organizations, since it is more continuous rather than episodic, and thus relies less on a single snapshot of the organization’s safety culture, which could be different in six months’ time. The Stack process also helps answer a question that has dogged safety culture pundits for several decades, namely, what should happen between surveys?

It remains to be seen whether the safety culture surveys carried out as part of the Future Sky Safety program remain an interesting experiment or become a watershed and platform for the rest of the aviation industry to build upon, working toward a more sustainable and explicit approach to managing and improving safety culture. Experience in ATM shows that even after evidence of compelling benefits, it can take years before such new initiatives take off and their use becomes more widespread. In the meantime, the organizations already involved will continue their safety culture efforts, and for those that want to begin, the tools are available.

Barry Kirwan is safety research coordinator, Eurocontrol; T.W. Reader is associate professor, Department of Psychological and Behavioural Science, London School of Economics; Anam Parand is researcher, Department of Psychological and Behavioural Science, London School of Economics; Richard Kennedy is senior program manager, aviation safety, the Boeing Company; Corinne Bieder is head of Safety Security Research Program, Airbus; Sybert Stroeve is senior scientist, Safety Institute, Netherlands Aerospace Centre NLR; and Arjen Balk is safety consultant, Safety Institute, Netherlands Aerospace Centre NLR.

Acknowledgments

This publication is based on work performed in Future Sky Safety, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 640597. Any dissemination reflects the authors’ view only, and the European Commission is not responsible for any use that may be made of the information it contains.

The authors thank the researchers and organizations participating in the project Future Sky Safety P5 “Resolving the Organisational Accident:” Eurocontrol, Airbus, Boeing, Deep Blue, ENAV (the Italian ANSP), KLM, London School of Economics, Trinity College Dublin and the Netherlands Aerospace Centre NLR.

References

Mearns, K.; Kirwan, B.; Reader, T.W.; Jackson, J.; Kennedy, R.; Gordon, R. “Development of a Methodology for Understanding and Enhancing Safety Culture in Air Traffic Ma agement.” Safety Science Volume 53 (2013): 123–133.

Reader, T.W.; Noort, M.C.; Kirwan, B.; Shorrock, S. “Safety Sans Frontieres: An International Safety Culture Model. ” Risk Analysis Volume 35 (2015): 770–789.

---

Featured image: © Idey | Adobe Stock
KLM airplanes: Saschaporsche | Wikimedia CC-BY-SA 3.0
One page procedure: Luton Safety Stack
Survey on tablet: © ZinetroN | VectorStock