That’s FRAT

Evolution of Business Flight Risk Assessment

Flight risk assessment tools (FRATs) have shown their value, and indications are that further refinement of the assessment process will provide even better results, according to a study conducted by the VanAllen Group and presented at the 2012 Corporate Aviation Safety Seminar.¹

This was no academic exercise. “The goal of the study was to accelerate the evolution of FRATs,” the author’s paper says. It describes a second generation of FRATs that is available in the marketplace: “They are the software evolution of the first generation hard-copy models [paper forms] that were integral elements of the initial safety management systems protocols and resources.”

Ten major non-commercial aviation departments participated, representing “high end” operations, most operating both domestically and internationally. Turbojets were the only type of business aircraft involved. Twenty percent of the FRATs were created by their own participants, and the rest used commercial FRATs from five different vendors.

The study collected data and participant responses for flights from August 2011 through January 2012. Because of the variety in FRAT metrics and scoring schemes among operators, it was decided to use the requirement for a “management review” — given a score of 100 percent — as the basic criterion (Table 1).²

“For the first three months, we gathered baseline FRAT and trip-leg data,” the paper says. “This created an average FRAT score for each participant. … At the midpoint, we asked the participants to deliberately seek to lower [improve] their scores throughout the second half.”

Average FRAT scores were collected at the end of the study. No overall lowering was found, however (Table 2).

The FRAT scores at times seemed counterintuitive. Some apparently higher-risk flight segments did not meet the “management review” hurdle:

• “Three training flights, among the highest-risk events in business aviation, averaged a score of 30 percent [less than one-third of the score that would have triggered a management review].”
• “Three hundred eighteen ferry flights, which have an accident rate four times greater than passenger-bearing legs, averaged a score of 44 percent, with only 7 percent of the flights reviewed.”
• “A medium jet on its international factory delivery flight, [with] no mentor pilot [and] no translator, was flown on a 14.7-hour duty day. The crew was the owner and his chief pilot.” The FRAT score was 65 percent.
• “The same medium jet, with the same crewmembers with less than 50 hours in type, conducted a night landing into a 4,000-ft [1,219-m] runway at sea level.” Again the FRAT score was 65 percent.
• “It is obvious that FRAT scoring is not an exact science,” the paper says.

The numbers may have told one story — or no story — but the participants told another. “The majority of the participants reported strong positive benefits gained from the FRAT process,” the paper says. “They also indicated the benefits continued to increase throughout the study.”

After its conclusion, the researchers conducted a meeting and conference call with all the operators. When asked the most important benefit they had gained from using a FRAT, participants offered responses such as these (paraphrased and summarized):

• “It forces crews to talk about trip issues. It is a teaching and learning tool.”
• “We found a number of FRAT conversations led to ‘Aha!’ insights.”
• “We now have historic data, not just lore, to help us modify our training to make it much more meaningful.”
• “Our crews shifted from ‘checking the boxes’ to truly understanding our operations, the risks incurred and how to most effectively manage or mitigate those risks.”
•  “We transitioned from using a FRAT for International Standard for Business Aircraft Operations (IS-BAO) compliance to it being a useful tool for measuring and managing safety.”

The FRAT is valuable, the paper says, not only for the information it provides but also for the process itself, raising awareness and changing attitudes.

Participants were asked to reply to additional questions about their FRATs. For example, “How important is your FRAT to your SMS [safety management system], on a scale with 5 = critical to 0 = not at all?” The average response was 4.2. Another question was, “How effective is your FRAT as a risk management tool, on a scale with 5 = extremely to 0 = not at all?” The average response was 4.0.

The average number of data points assessed by users’ FRATs was 51 (with 90 the largest number). The number of human factors data points tracked ranged from five to 18, with an average of 10.

“That translates to about 20 percent of data points focused on human factors,” the author told ASW. “Yet, about 70 percent of accidents and incidents are human factors–sourced. This disparity underscores the need for more comprehensive development of FRAT data points.”

The paper forecasts that FRATS in the not too distant future will be completely integrated with flight operational quality assurance programs, and with as many data points as possible automated.

“The crew would have current flight risk ratings as a cockpit readout, with risks and mitigations displayed upon request,” the paper says. “When a significant change in the risk occurs, a message would be displayed with mitigation recommendations listed. A parallel message could be shared with management. … Variances would be recorded, reported and discussed. Procedural intentional noncompliance events, a major factor in accidents and incidents, would become much less frequent.”

The next generation of FRATs will be much more effective, the paper says.

Fatal accidents numbered 42 in 2012, 17 percent more than the 2007–2011 average (Table 4). Six of those 42 involved commercial aviation airplanes, and five involved commercial aviation helicopters.  Sixty-three fatalities resulted from the 2012 accidents, a decrease from 67 in 2011 and an average 66 in 2007–2011. There were 50 serious injuries in 2012 accidents, compared with 44 in 2011 and an average 48 in 2007–2011.

“In 2012, a total of 636 incidents were reported,” the TSB said. “This is a 6 percent decrease from the 2011 total of 677 and a 21 percent decrease from the five-year average of 808.” 

Notes

1. Agur, Peter v. Jr. “Second Generation FRATs: Strengths, Weaknesses and Next Generation Opportunities.” Flight Safety Foundation, Proceedings of the 57th annual Corporate Aviation Safety Seminar.
2. For example, at one operator a score of 26 would prompt management to review the trip leg, while a score of 19 might have the same result at another operator. For the sake of apt comparison, both would be scored equally by the researchers as 100 percent.
3. TSB. “2012 Statistical Highlights: Aviation Occurrences.”