Reports
Event Limits
Flight Data Monitoring Based Precursors Project: Part 1 — Runway Excursions
U.K. Civil Aviation Authority (CAA) Safety Regulation Group. Report 2012/01. 28 pp. December 2012.
The concept — although not necessarily the practice — of FDM is simple enough. It requires aircraft equipped to access and record selected parameters of flight data, which can then be downloaded via quick access recorders (QARs) or wireless real-time transmission. The data can be analyzed from individual flights and aggregated from numerous flights, usually in a deidentified form, to give an overview of exposure to risk. That, in turn, gives operators a handle on mitigating identified risks.
The report describes a project undertaken by the CAA to determine the effectiveness of FDM in helping operators monitor and reduce the risk of landing runway excursions, identified by the CAA and the industry as one of “seven significant safety issues.”1 Flight Safety Foundation has also emphasized the importance of minimizing runway excursions, collaborating with the industry in a Runway Safety Initiative that resulted in the production of the Runway Excursion Risk Reduction (RERR) Toolkit.
The CAA project was developed in connection with Aerobytes, a supplier of FDM software, and an unidentified airline referred to only as the operator. In the report’s preface, Aerobytes comments: “As any experienced user of an FDM/FOQA system will tell you, developing theoretical high-level/low-detail ‘concept’ analysis solutions is easy. The challenge is to translate those ‘concepts’ into reliable and practical methods that will work across a range of aircraft — aircraft which won’t necessarily all record the ‘perfect’ set of parameters.” The company says that its programming philosophy is to simplify the combination of parameters selected for analysis, consistent with utility, and minimize dependency on what it calls “exotic” parameters that many aircraft types are not equipped to provide.
The operator, in turn, says that “it has been challenging to translate safety data into consistent measures against specific risks. We experienced that monitoring hundreds of FDM events and event descriptors from the safety reporting database can be a very time-consuming exercise and may produce varied analysis.”
For the project, the operator says, “Variables and event logics were modified to confirm consistency and accuracy of the data. The sole aim of the project was to identify FDM-based precursors which could easily be adopted into any FDM system, but will provide sufficient information to assess the exposure to the runway excursion risk. Although our FDM vendor (Aerobytes) had provided us with some very useful algorithms to monitor key values and events, this project helped in identifying some finer improvements which could further enhance the analysis of the data.”
The FDM data were obtained from the QARs of several of the operator’s Airbus A320s, based on 587 flights during summer and 250 flights in one winter month. In the trial, the approach and landing phases (what Aerobytes calls “states”) of flight were studied. The report says, “The [Aerobytes FDM] program sets an end point at touchdown and then looks backwards through the data until the gear and flaps are up, which is set as the start of the approach state. … The landing state is the period between touchdown until the end of rollout. This is defined as after 90 seconds, or [when] groundspeed is less than 50 kt or there is a heading change of more than 20 degrees.”
Twenty values were selected for analysis. For approach, they included height above airfield (AAL) at various stages, such as becoming established on the glideslope and gear selected down, and airspeed versus the aircraft-computed reference landing approach speed. Landing values included data such as airspeed at touchdown and time from touchdown to reverse thrust application.
Experience with the early data analysis led researchers to modify some of the criteria for a stabilized approach during the study. For example, the CAA noted that precision approaches and non-precision approaches2 are so different that “it is important to determine the type of approach being flown and obtain comparable metrics from both types of approach.”
Based on the project’s FDM data, the report identified “significant” events considered potential precursors to a runway excursion and recommended that operators use those event limits in determining stable approach criteria. It warned, however, that extreme outliers in calculated data distributions are suspect, and that “each of these significant events must be fully validated so as to remove false ‘events.’ In this way the workload associated with each measure will be minimized, whilst assuring data quality.”
The recommended precursor event limits are as follows:
- Unstable approach: Below 1,000 ft AAL or below 500 ft AAL — the lowest height AAL at which the approach was unstable in instrument meteorological conditions or visual meteorological conditions, respectively.
- Long flare: Distance greater than 2,000 ft (610 m) from reaching flare height to touchdown.
- Long landing: Distance greater than 2,500 ft (762 m) from runway threshold to touchdown.
- Fast landing: Airspeed at threshold greater than VAPP (final approach speed calculated by Airbus aircraft) or VREF (reference landing speed for non-Airbus aircraft).
- Runway remaining at touchdown: Less than 4,000 ft (1,219 m).
- Once the standardised precursor measures have been implemented, thought must be given to the aggregation, analysis and presentation of the results,” the report says. “For example:
- Measures of exposure by airfield, runway, fleet.
- Frequencies/probabilities of events by airfield and runway.
- Values and context data (e.g., airfield, runway, type of approach) for each event.
“[These] data should be output in a standard database/spreadsheet format to allow further analysis and also aggregation with other operators’ data, if agreed.”
Weighed on the Bioscale
Development, Validation, and Fairness of a Biographical Data Questionnaire for the Air Traffic Control Specialist Application
Dean, Michelle; Broach, Dana. U.S. Federal Aviation Administration (FAA) Civil Aerospace Medical Institute (CAMI). DOT/FAA/AM-12/19. 20 pp. December 2012.
As it happens, the FAA has long included biographical data, or “biodata,” in its assessment of applicants. “The Federal Aviation Administration (FAA) has conducted [biodata] investigations of the ATCS occupation,” the report says. “Following the 1981 controller strike, the FAA faced an enormous organizational challenge in rebuilding this highly technical workforce. While the core of the post-strike ATCS selection process from 1981 through the mid-1990s was a cognitive aptitude test battery, researchers at CAMI investigated alternative assessments, including biodata. Two instruments in particular were administered to several thousand newly hired air traffic controllers for research purposes between 1981 and 1992: the Applicant Background Assessment (ABA) and the Biographical Questionnaire (BQ). Research with these instruments indicated that biodata had promise as a personnel selection tool for the ATCS occupation.”
The FAA developed three versions of the “biodata scale”: with 80 items, 100 items or 120 items.
As the report points out, the stakes are high for both applicants and the FAA. Many applicants spend thousands of dollars for tuition and fees in the FAA’s Air Traffic Control Collegiate Training Initiative, hoping to be hired by the FAA. For the agency, the report says, “ATCS training is intensive, extensive and expensive. Completion of all training phases takes an average of two to three years, depending on facility assignment. Failures waste FAA training dollars and personnel resources. They also result in fewer people becoming controllers, a critical concern for the agency as the post-strike generation of controllers reaches retirement age.”
Researchers used several techniques to rate the validity of the biodata questionnaires, such as looking for correlations among the ABA, the BQ and average supervisory ratings. The biodata questionnaires also were measured against the computerized Air Traffic Selection and Training (AT-SAT) aptitude test battery composite score in predicting the supervisory ratings.
The report concludes that “each version of the biodata scale had significant incremental validity over the AT-SAT composite score, accounting for 29 percent to 32 percent additional variance in average job performance ratings.” Second, “the 80-item version was more efficient (fewer questions for about the same statistical gain) than either the 100- or 120-item versions of the biodata scale.”
Notes
- The other “significant seven” safety issues were airborne conflict, airborne and post-crash fire, controlled flight into terrain, loss of control, ground handling and runway incursion/ground handling.
- A non-precision approach was defined, adopting International Civil Aviation Organization Annex 6 criteria, as “an instrument approach and landing which utilizes lateral guidance but does not utilize vertical guidance.” A precision approach was defined as “an instrument approach and landing using precision lateral and vertical guidance with minima as determined by the category of operation.”