Aviation accident fatality data for 2008–2012 tell an encouraging and improving story for countries advancing the furthest on the development continuum. At the same time, however, improvement is not being seen among the least developed states, which are falling behind the rest of the world when accident fatality data is examined.
To understand the changes, let us review some background.
Four National Categories
In a study published in 2010 concerning data from 2000–20071 on the death risk for passengers on scheduled commercial flights, I divided the world’s nations into four categories:
- The traditional “First World,” which included the British Commonwealth members Australia, Canada and New Zealand; Israel; Japan; the United States; and Western Europe.2
- The “Nouveau Riche” countries that had achieved First World standards of life expectancy and/or gross domestic product per capita by the early 21st century. These countries were Bahrain, Bosnia-Herzegovina, Brunei, Cyprus, Czech Republic, Hong Kong, Jordan, Singapore, Slovenia, South Korea, Taiwan, and the United Arab Emirates.
- The “Newly Industrialized” countries that, in surveys of economists, were described as “having economies more advanced and developed than those in the developing world, but not yet (showing) the full signs of a developed country.” These countries were Brazil, China, India, Malaysia, Mexico, Philippines, South Africa, Thailand, and Turkey.
The “Less Developed” nations not in one of the three previous categories.
To assess passenger safety during 2000–2007, the metric I used was death risk per flight (ASW, 11/07). It was the answer to the question: “If a passenger chose a flight at random from among those of interest (e.g., scheduled Brazilian flights), what is the probability that he or she would perish in an accident?” I argued that this metric was superior to others that are sometimes used, even to the apparently similar metric passengers killed divided by passengers carried.
The data revealed that there was a large difference in death risk per flight in the four categories of nations (Table 1).3
Because the results for the Nouveau Riche and Newly Industrialized nations were almost identical, it seemed reasonable to merge the two groups into a single category, which was called Advancing nations (Table 2). For the present article, although the three categories remain the same, to increase consistency in nomenclature “First World” has been renamed “Fully Developed,” and “Less Developed” has been renamed “Least Developed.”
Statistical Significance
The first question a statistician would ask about Table 2 is whether its differences are statistically significant, that is, if observed differences are so large that chance could not plausibly explain them.
Calculations made clear that the cross-national categorical differences in Table 2 are of very high statistical significance. To say that they reflect random variations in luck would stretch the laws of probability to their breaking point.
A Surprising Outcome
A puzzling aspect of the 2000–2007 findings was that a group of Advancing nations, which were approaching Fully Developed standards of economic development and/or life expectancy, nonetheless had aviation safety records that were considerably closer to those among Least Developed countries than to those of Fully Developed countries.
Nation-by-nation findings of the sociologists Hofstede and Hofstede indicated that, on the dimensions “power distance” and “individualism,” the numerical scores of Advancing nations were almost identical to those of Least Developed nations, while very far from those of the Fully Developed ones.4,5 While I did not want to get “too caught up in speculation,” I wondered whether the explanation for the safety findings was that the economic progress among Advancing nations had not been accompanied by a corresponding cultural shift.
The 2008–2012 Record
After the crash of Asiana Flight 214 at San Francisco in July 2013, there was much discussion about whether Korean culture may have played a role in the accident.6 While this speculation may have been unwarranted, I was not in a position to discount it, given the results above concerning 2000–2007.
Yet the rapidly changing domain of aviation safety creates special relevance to the saying, “That was then, this is now.” In the five years 2008–2012, there were approximately 150 million scheduled commercial passenger flights worldwide. Information about these 2008–2012 flights allows an exploration of whether patterns that applied at or before 2007 continue to prevail. Now, in addition to making a comparative risk assessment for Fully Developed and Advancing nations, I will estimate absolute risk levels over 2008–2012 both globally and in various groups of nations.
Two Safety Metrics
Before presenting results for the years 2008–2012, it is worth revisiting the issue of how best to estimate passenger mortality risk for air transportation. The death risk per flight statistic pays no attention to the length or duration of flights. That feature seems reasonable because the overwhelming majority of commercial aviation accidents occur during the takeoff and climb phases or the descent and landing phases of flight. However, the simple statistic passengers killed divided by passengers carried also embodies that feature.
The problem with this simpler formula is in the use of the number of deaths as its numerator. I have argued that if an airliner hits a mountain, killing all passengers, the implications about safety are the same whether 50 passengers or 150 are aboard. And a crash that kills 28 out of 280 passengers is not the same as one that kills 28 out of 28. (In the former case, excellent performance of emergency procedures may have saved 90 percent of the passengers.) Statistics that weigh crashes by their numbers of deaths, in other words, are vulnerable to irrelevant fluctuations in the fraction of seats occupied, yet insensitive to salient variations in the passenger survival rate.7
Because the statistic death risk per flight weighs each crash by the percentage of passengers killed, it avoids that problem. Yet death risk per flight achieves this advantage by assuming that the hypothetical traveler chooses a flight completely at random from among those of interest. But passengers do not choose flights at random: The average Airbus A380 has far more passengers than the average Beechcraft 1900. If there is any correlation between the size of an aircraft and its risk of crashing, then a statistic that ignores aircraft size could yield biased and misleading results about the risks to passengers. This shortcoming of death risk per flight could partially or even totally negate the case for its superiority.
Both death risk per flight and passengers killed divided by passengers carried suffer from weaknesses, and there is no obvious way to gauge the relative importance of their imperfections. Under the circumstances, a sensible strategy would be to perform and report calculations with both of them. Taken together, the two statistics might offer a perspective that is more reliable than either one on its own. That is the approach taken here.
The Recent Data
Risk estimates can now be made about the last five calendar years (Table 3). Using the numbers in Table 3, death risk per flight estimates for 2008–2012 can be calculated for three groups of nations and the entire world (Table 4). This enables comparing these estimates with the corresponding data for 2000–2007.
Major divergences are evident, both within the 2008–2012 risk estimates and in the changes compared with the 2000–2007 counterparts. The worldwide mortality risk for air travelers was almost the same in the past five years as in the previous eight (1 in 3.3 million versus 1 in 3.0 million). But the Advancing nations cut their risk by a factor of 3.3 (from 1 in 2 million to 1 in 6.6 million), while the Least Developed nations showed no improvement at all. The Fully Developed nations improved somewhat and remained the safest group of nations under this risk metric.
The Advancing nations’ mortality risk fell between that of the Least Developed and the Fully Developed groups, both in 2008–2012 and 2000–2007. But within that range, this group’s statistical risk changed considerably from the earlier to the later period. During 2008–2012, Advancing nations’ risk statistic was 3.0 times as high as that in the Fully Developed group, compared with 7.0 times as high during 2000–2007. And their 2008–2012 statistic was a factor of 11.0 lower than the Least Developed statistic, compared with a factor of 2.5 lower in the prior eight years.
In relative terms, therefore, the safety performance of the Advancing nations moved from far closer to the Least Developed group to far closer to the Fully Developed group. This movement continues a trend that began before 2000. In a previous study, for example, a colleague and I found scant differences between safety performance of the Advancing nations and the Least Developed during 1987–1996.8
Moreover, the difference between the 2000–2008 risk for Advancing nations and Fully Developed nations was not statistically significant at the usual 5 percent significance level. Because the Advancing nations represented 27 percent of the combined Advancing/Fully Developed nation flights during 2000–2008, an equal-safety model would posit that, when one of those nations suffered a fatal air crash during that period, the chance is 27 percent that it would be an Advancing nation.
Under that probability model and given the rarity of fatal crashes, it is not statistically abnormal that the Advancing nations had as many of the combined group’s full-crash equivalents over 2000–2008 as they actually did. This outcome is in sharp contrast with statistical test results for earlier periods, which emphatically rejected the equal-safety model.
Less encouraging is the finding about the Least Developed nations. The data do not show improvement in their death risk per flight statistic between 2000–2007 and 2008–2012 (though the step backward in the data — from 1 in 800,000 to 1 in 600,000 — is not statistically significant). This happened even though the Least Developed nations had considerably more room to improve than the other national groups. That these nations have fallen further behind other nations highlights the importance of continuing or expanding the initiatives for safety in those regions.
Death Risk per Boarding
An alternate way to estimate differences among nation groups is death risk per boarding (Table 5). This statistic is based on the question, “Suppose that the boarding passes for all passengers carried during 2008–2012 among the airlines of interest were assembled, and one of the boarding passes was chosen at random. What is the probability that its bearer perished in an aviation accident?”
Because there are many more such boarding passes issued to passengers flying on widebody jets than to those flying on small, piston-powered commercial aircraft, this statistic does take aircraft size into account. It is actually the passengers killed divided by passengers carried ratio discussed earlier.
It is striking in Table 5 that worldwide death risk per boarding was 40 percent lower than death risk per flight (1 in 5.5 million versus 1 in 3.3 million). The difference arises because the airplanes involved in fatal accidents carried fewer passengers on average than airplanes in general. That pattern did not hold in the Fully Developed nations, but very much did among both Advancing and Least Developed nations.
Table 5’s biggest change from Table 4 is that the death risk per boarding for Advancing nations has declined by a factor of 2.5, from 1 in 6.6 million to 1 in 16 million. That happened because, within Advancing nations, the airplanes involved in fatal crashes had on average 62 passengers aboard, versus an average of 115 for all flights. This difference is not weighed in calculating death risk per flight, but is highly consequential in computing death risk per boarding.
In relative terms, Table 5 indicates that the Advancing nations have now all but caught up to the Fully Developed ones. Among passengers who boarded planes in the Fully Developed nations, approximately 1 in 18 million died in accidents. For the Advancing nations, the statistic was 1 in 16 million. By this metric, the death risk in Advancing nations is only 1.1 times that in the Fully Developed nations. That difference, like the corresponding difference for death risk per flight, is not statistically significant. In other words, under both risk metrics, the difference between the Advancing and Fully Developed nations has diminished to the point that it can be attributed to chance fluctuations.
Convergence at the Top?
It follows that, before anyone cites cultural or speculative “explanations” about why Advancing nations cannot match the aviation-safety successes of Fully Developed counterparts, we should bear in mind that they nearly did so in 2008–2012. If they have not completely closed the gap, they have reduced it enough to raise doubts that it continues to exist.
In view of the reaction to the crash of Asiana Flight 214, it is worth noting that South Korean air carriers sustained no passenger deaths at all during 2008–2012. Even if one stretches the observation period to include the San Francisco accident in July 2013 — which is statistically inappropriate — both death risk per flight and death risk per boarding on South Korean carriers have been below 1 in 75 million since the beginning of 2008. No national group in Tables 4 or 5 has outperformed this record.
Reducing the Gap
Perhaps, like the dog that didn’t bark in the Sherlock Holmes story, what is most important about recent safety statistics in commercial air transport is what didn’t happen. The airlines of the Least Developed nations are not unsafe, but their mortality risk levels have not visibly improved recently, and these levels could drop by an order of magnitude if those nations achieved in the future what other nations have already achieved. The hope is that, in future data analyses, the shrinkage of the gap that we have seen between Advancing and Fully Developed nations will also materialize between the Least Developed nations and the rest.
Arnold Barnett, Ph.D., is the George Eastman professor of management science and professor of statistics at the Massachusetts Institute of Technology. He received the FSF President’s Citation in 2002 for “harnessing the power of innovative statistical analysis to build conceptual bridges between data and significant aviation safety issues so that they can be readily understood.”
Notes
- Barnett, A. “Cross-National Differences in Aviation Safety Records.” Transportation Science, Volume 44(3), pp. 322–332. 2010.
- Western Europe, in the earlier study and the present study, comprises Austria, Belgium, Denmark, Finland, France, Germany, Greece, Iceland, Ireland, Italy, Luxembourg, the Netherlands, Norway, Portugal, Spain, Sweden, Switzerland, and the United Kingdom.
- Although any accident is attributed to the national origin of the airline involved, calculations showed that attribution of the accident to the country in which it occurred would yield nearly identical results.
- Hofstede, G. and G. J. Hofstede. Cultures and Organizations: Software of the Mind. McGraw-Hill, New York. 2005.
- Power distance relates to the degree of deference shown to people of higher authority. In the cockpit, large power distance could translate into strong reluctance to challenge the captain.
- Individualism relates to the ability to take standard rules as guides but to feel free to go beyond them when it seems desirable (i.e., to improvise when a situation is not literally covered by the rules). In aviation, that mental flexibility can be important in coping with an emergency.
- For example, “Could Malcolm Gladwell’s Theory of Cockpit Culture Apply to Asiana Crash?” National Geographic News, July 9, 2013; “Did Korean Culture Contribute to Asiana Crash in San Francisco?” Japan Times, July 10, 2013; Bertorelli, Paul, “More on Asiana 214,” AVweb, July 11, 2013.
- Barnett. “Aviation Safety and Security.” Chapter 11 in The Global Airline Industry. John Wiley and Sons. 2009.
- Barnett, A. and A. Wang. “Passenger-Mortality Risk Estimates Provide Perspectives About Airline Safety.” Flight Safety Digest April 2000, pp. 1–12.