Evaluation of a pilot’s speech before a flight might prove to be an accurate means of determining whether he or she is dangerously sleepy or fatigued1 and should not fly an aircraft, according to a report by a team of Brazilian researchers.2
The report, published in the April issue of Aerospace Medicine and Human Performance, based the conclusion on analysis of the communication between a pilot and controllers in an air traffic control tower just before a fatal accident.
The report did not discuss many of the details of the accident flight, “due to ethical considerations,” but disclosed that the pilot was a 45-year-old Brazilian man with no preexisting disease and that flight operations before the 1000 local time crash were uneventful. There had been no reported problems with the airplane and no complaints from the pilot, the report added.
“Near the time of landing, the pilot committed a technical failure that caused the aircraft to tilt down, with no time to recover,” the report said.
Brazil’s Center for Investigation and Prevention of Aeronautical Accidents (CENIPA) made available to the researchers several audio recordings of the pilot. One was a private recording made while the pilot was off duty, another was a recorded conversation with the flight planning center 35 hours before the accident and a third was recorded during the accident flight. The report said that about 16 minutes of recordings were made the day before the accident and 30 minutes were taken from recordings on the day of the crash, including “dialogues from the taxiing of the aircraft and the preparation for takeoff until the last communication with the air traffic controller.”
Acoustic and statistical analyses compared the pilot’s speech characteristics on the day of the accident with those of the previous day. Those characteristics, also called acoustic parameters, included:
- Total articulation time — which the report said was “composed by ‘fluent syllables’; that is, speech without disfluencies [involuntary disruptions] such as silent pauses, repetitions, prolongations, hesitation, etc.” — and articulation rate — which the report defined as the number of syllables spoken, divided by the total articulation time;
- Total pause time, including not only silence but also disfluencies such as repeated words;
- Elocution time, or total speech duration, and elocution rate — or the number of syllables divided by the total elocution time;
- The number of pauses and average length of those pauses; and,
- Percentage of disfluency, or the number of silent pauses, repetitions, prolongations and similar traits in relation to the total number of spoken syllables.
Acoustic parameters on the day before the accident differed from those noted later, before and during the accident flight, the report said.
The pilot’s elocution and articulation rates were slower the day of the accident, the report said, adding that when his speech patterns that day were compared with speech patters for men from the same part of Brazil — Minas Gerais state — his speech was slower than theirs.
“There was also a significant increase in the disfluency of the speaker on the day of the accident, in comparison with the previous days, namely in days off at home and in the same working conditions 35 hours before the plane crash,” the report said. “This is additionally evidenced by an increase in silent and filled pauses.”
The report also noted “interesting voicing … aspects” on the day of the accident, including “vocal fry,” in which a speaker uses his or her lowest tone of voice, which sounds creaky and breathy. Vocal fry is unlikely to occur in situations involving high levels of background noise, such as that present in an airplane cockpit, the report said.
These characteristics of a pilot’s speech often result from fatigue, the report said.
“Changes in the temporal organization of speech that make speech slower are triggered by the state of fatigue,” the report said. “These changes occur in fatigue/sleepiness because in this situation, there are changes in the patterns of muscular contraction and in the neurological speech commands.
“Considering the … variations in the speech parameters of the pilot on the day of the accident (changes in temporal organization, disfluency and voicing aspects), compared with the day before the accident and also speech normality patterns, it is concluded that the types of speech and voice analyses adopted in this study are promising for the detection of fatigue and sleepiness.”
The report acknowledged that speech and voice analysis constitute an indirect measure of fatigue and sleepiness. Nevertheless, the document added, other indirect measures such as the evaluation of a pilot’s eye movements or electrical brain activity are “invasive for the pilots, more difficult to carry out and less feasible than speech recording and analysis.”
The use of speech and voice analysis to identify fatigue would be most promising in situations in which pilots are in frequent communication with other crewmembers or with air traffic controllers, the report said.
“The analyses discussed in this paper can be used to mitigate accident risks by assessing pilot conditions before and during flights,” the report added. “For such, an automatic analysis of the elocution and articulation rates and of the pauses produced during speech can be done in real time. A voice activity detector can be used to detect pauses, the patterns of which will be processed. Speech recognition techniques can be adapted to measure elocution and articulation rates. Parameters associated with voice quality require a more sophisticated analysis to detect, for example, the occurrence of vocal fry. Last, but not least, full-scale validation studies are needed to determine whether the results obtained in the present case report can be replicated over individuals and in other languages.”
Note
- The report differentiated between mental fatigue, which it defined as “the state of performance impairment following a period of mental or physical effort,” and sleepiness, “the transition between wakefulness and sleep and … the tendency to fall asleep even when the individual is supposed to be active.”
- deVasconcelos, Carla Aparecida; Viera, Maurilio Nunes; Kecklund, Goran; Yehia, Hani Camille. “Speech Analysis for Fatigue and Sleepiness Detection of a Pilot.” Aerospace Medicine and Human Performance Volume 90 (April 2019): 415–418.
Featured image: © Sergey Nivens | Adobe Stock
Pilot with headset: © Jacob Lund | Adobe Stock
