The current system of evaluating the severity of runway incursions, which emphasizes the horizontal/vertical distance between the aircraft (or other entities) at their closest point, disregards the overwhelming majority of incursions that are less severe but result from similar errors, according to a study by U.S. and Dutch researchers.1
The study — conducted for the U.S. Federal Aviation Administration (FAA) by researchers from the FAA and the Netherlands Aerospace Centre NLR — concluded that a better method of evaluation would focus on the risks of various scenarios that occur at the beginning of a runway incursion.
Reporting and analysis of runway incursions are critical elements in the safety management of runway operations, the researchers said in their report on the study, published in the Reliability Engineering and System Safety journal.
Traditionally, runway incursions2 are classified according to five severity categories defined by the International Civil Aviation Organization (ICAO):
- Category A includes the most serious events, “in which a collision was narrowly avoided.”
- Category B events are those in which “separation decreases and a significant potential for collision exists, which may result in a time-critical corrective/evasive response to avoid a collision.”
- Category C events are those “characterized by ample time and/or distance to avoid a collision.”
- Category D events are those that fit the definition of a runway incursion, “such as incorrect presence of a single vehicle/person/aircraft on the protected area of a surface designated for the landing and takeoff of aircraft but with no immediate safety consequences.”
- Category E includes events that cannot be classified because of insufficient or conflicting information.
In their report, the researchers analyzed data from the FAA’s Aviation Safety Information Analysis and Sharing (ASIAS) program, which uses ICAO’s recommended Categories A through D, but not Category E.
Within the database, the severity categorization is based largely on “the particular outcome of a runway incursion,” especially the closest distance of the aircraft, ground vehicle, person or other entity involved, the report said.
However, the document added, “This closest distance attained depends to a considerable extent on uncontrolled random circumstances, such as another aircraft being nearby at the time of the initiation of the runway incursion. In incursions that are judged as being less severe [those in Categories C or D, for example], typically the same types of errors or misunderstandings by pilots or controllers lead to initiation of runway incursions, and the distinction with more severe [Categories A and B] cases is primarily due to some uncontrolled circumstances. The consequence is that current safety management is driven largely by random outcomes, wherein lessons from incursions with less severe (C, D) outcomes may be undervalued, and there may be an overreaction to severe (A, B) outcomes.”
A New Framework
The new framework suggested by the authors “does not use an outcome-based severity category,” they said. Instead, it is “strictly based on the risk of scenarios associated with runway incursions.”
Their report added, “There are many ways in which a runway incursion can arise, and given its initiation, there are many ways in which it can develop next, up to an accident as the most severe consequence.”
As an example, the report described a situation in which one aircraft is taxiing from a gate for departure and a second aircraft is approaching a runway for landing. An analysis of the situation includes a look at the initial state of the aircraft, before any precursor of a runway incursion occurs, and takes into account such factors as the sizes of the aircraft and their positions at the gate and along the approach path. Various events can occur that could be precursors to a runway incursion, such as the pilots of the first aircraft making a wrong turn that affects their situational awareness or the pilots of the second aircraft forgetting an air traffic control (ATC) instruction and then losing situational awareness.
“Such inflicted states may lead to a runway incursion event … wherein [the first aircraft] passes the hold-short line of the runway and comes into conflict with [the second aircraft] that is about to land,” the report said. “Often, however, these kinds of inflicted states do not lead to a runway incursion, as there are various events that can prevent the progression to a runway incursion event, such as the pilots recognizing that they are at a wrong position or being warned by ATC. …
“All these types of events, their orderings and timing have impact on the kind of final state in the evolution of the runway incursion at [the time] when the entities involved are closest.”
For example, the report said, the final states may be that the second aircraft flies over the first aircraft at 100 ft (31 m), the second aircraft goes around at 1 nm (2 km), the first aircraft stops 10 ft (3 m) before the runway while the second aircraft passes, or the two aircraft collide.
Severity-Based Assessments
Under the current system of severity-based assessments of runway incursions, the assessment typically is made by a team made up of air traffic controllers, airline pilots and airport operation/design experts, along with a team leader.
ICAO’s guidelines call for the team to base its assessment on six factors, including the closest horizontal distance between the aircraft and/or vehicle involved and the “geometry of the encounter” — with encounters between entities on the same runway considered more serious than those involving one aircraft flying toward the runway and the second entity on the ground.
The other factors are:
- The extent of evasive or corrective action;
- The available reaction time;
- Weather, visibility, surface conditions and other environmental conditions; and,
- Factors affecting system performance such as communication system failures.
In ASIAS data for reported runway incursions from Oct. 1, 2008, through Sept. 30, 2013, each year there were two to 13 of the most serious events (Categories A and B); during the same period, there were about 400 Category C incursions and 600 Category D incursions.
The report used the ASIAS runway incursion data and accompanying event narratives in its examination of the closest proximity attained by the aircraft and other entities involved but noted that distance estimates were not consistent. Nevertheless, the report said that in Category A incursions, “either the vertical distance was close to zero, i.e., both entities were on or very close to the ground (18 incursions), or the horizontal distance was close to zero (22 incursions).” In eight of those cases, a collision was narrowly avoided. Overall, the closest proximities for Category A incursions generally were within 60 m (197 ft) horizontally and 40 m (131 ft) vertically.
Category B incursions involved the smallest horizontal distances, “which mostly represent flyovers,” the report said, and vertical distances up to 120 m (394 ft).
For most Category C incursions, the closest distances were between 300 m (984 ft) and 2,500 m (1.3 nm), “reflecting distances where aircraft canceled takeoff or initiated a go-around,” the report said.
For Category D incursions, typically involving a conflict with a landing aircraft, the closest horizontal distance was more than 1,280 m (4,200 ft).
“The usage of the current severity categorization has led to a large distinction between the frequency of severity A and B incursions (1.2 percent) … and the frequency of severity C and D incursions (98.8 percent),” the report said. “As safety management tends to perceive A and B outcomes as more problematic than C and D outcomes, the large distinction in the statistics implies that lessons from the large majority of severity C and D incursions are undervalued in the safety management cycle and that there may be an overreaction to severe (A, B outcomes).”
The greatest limitation of this method of assessment, as noted, is its dependence on “uncontrolled random circumstances,” the report said, citing as an example a selection of four runway incursions that involved aircraft that had been lined up on a runway without ATC clearance. The severity categories varied, however, depending on “the random circumstance [of] whether a landing aircraft was close to the runway at the time of the incursion.
“The type of error made was the same, but the severity was either A, C or D. In other words, if a landing aircraft would have been nearby in incursions 3, 5 or 6, then the severity could well have been A or B.”
In the case of one of the Category D events, the report added, “it could even be argued that, other conditions being equal, the risk associated with the behavior … is highest, since the pilot first lined up without a clearance and next initiated takeoff without a clearance, thus creating two possibilities for a conflict.”
The severity-based evaluations also do not “provide means to structure reasons [for] the runway incursions and to evaluate the risk implications,” the report said.
Overcoming Limitations
Those limitations can be overcome by using a risk-based approach “for potential consequences, given the start of a runway incursion,” the report said. “We distinguish between what did happen until the initiation of a runway incursion and what may happen following the initiation of the runway incursion.”
Examples of runway incursion scenarios include a small aircraft entering a runway from an intersecting taxiway while its pilots are lost and coming into conflict with a large aircraft taking off in good visibility, or pilots of a small aircraft crossing the approach end of a runway without ATC clearance and coming into conflict with a small aircraft landing in reduced visibility.
“Given a particular runway incursion scenario, it can be argued what its consequences may be and what the probabilities of these consequences are, i.e., what the associated risk is,” the report said.
The report added that use of this framework requires development of an inventory of runway incursion scenarios that describe runway conflicts between aircraft or between aircraft and other entities. Scenario descriptors consider such factors as runway configuration, types and operations of the involved aircraft or other entities and the intent of the human operators.
The sample inventory in the runway incursion study included 169 main scenarios. Among them are 61 scenarios involving conflict between an aircraft taking off and another aircraft, vehicle or person; 56 scenarios involving conflicts during landing; and 52 scenarios involving conflicts while an aircraft is lining up on a runway.
The authors tested the inventory using a dataset of 232 runway incursions that occurred from October 2003 through September 2010 and found that 98.7 percent could be “mapped to” (or classified according to) at least one of the main scenarios, and 31 percent could be mapped to multiple scenarios.
In the cases involving multiple scenarios, sufficient information was not available in the event narratives to select some of the scenario descriptors. Often, the missing information dealt with the intent of the pilot or vehicle driver whose actions led to the runway incursion.
Steps in the Framework
The report said the proposed risk-based framework consists of five steps:
- Mapping runway incursions to scenarios. This establishes a basis for risk-based assessment and must be completed for each runway incursion, “using only information up to its initiation.”
- Assessing the probabilities of various scenarios, expressed as rates per airport movement.
- Assessing conditional probabilities of a collision using risk modeling.
- Assessing “the conditional probabilities of the human and material collision impact categories, given a collision in a scenario.” Risk modeling also is a key component in these assessments.
- Evaluating runway incursion risk by combining the three previous steps and comparing the result with safety criteria.
The most important elements of the proposed framework are the inventory and the risk-modeling process used to evaluate the risk implications of the runway incursions, the report said.
“In the proposed risk-based framework, collision risk results have to be attained for each scenario, i.e., for each subcase per main scenario,” the report said. “Building on collision risk models in [earlier research], it was shown that agent-based dynamic risk modeling can well account for dependencies between runway incursion scenario descriptors, and it can systematically achieve collision probabilities for large sets of runway incursion scenarios.”
Information gathered through the framework could eventually be developed as a basis for risk assessment of airport operations and operations design, the report added.
“In such integral safety management, runway incursion events are safety indicators that are used to update probability estimates of runway incursion scenarios made in the design phase,” the document said. “The uptake of the new risk-based framework in integral safety management stands in contrast with the current severity-based evaluation of runway incursions, which focuses on their outcomes and has no risk-assessment component.”
The authors said that their findings should not be interpreted as meaning that information about the actual outcome of a runway incursion should not be considered. That information, dealing with “the ways that runway incursions evolve and end,” can be valuable in validating collision risk models, they said.
The report concluded that a risk-based framework also should be considered as a superior method of evaluating other types of air traffic occurrences — such as inadequate separation, deviation from an ATC clearance or unauthorized penetration of airspace.
“In the light of the identified limitations of the severity-based evaluation of runway incursions and the advantages of the proposed risk-based framework, we advise future research on the potential limitations of severity-based evaluation of other air traffic incidents and the possibilities for risk-based assessment for a range of air traffic scenarios.”
Notes
- Stroeve, Sybert H.; Som, Pradip; van Doorn, Bas A.; Bakker, G.J. “Strengthening Air Traffic Safety Management by Moving From Outcome-Based Towards Risk-Based Evaluation of Runway Incursions.” Reliability Engineering and System Safety Volume 147 (2016): 93–108.
- The study defined a runway incursion as “any occurrence at an aerodrome involving the incorrect presence of an aircraft, vehicle or person on the protected area of a surface designated for the landing and takeoff of aircraft.”
Featured image: © style67 | AdobeStock; otisabi | istockphoto.com
Interior image: © Alexander Gatsenko | istockphoto.com
