SESAR 3 JU members and partners have recently investigated the impact that automatic speech recognition (ASR) can have on flight efficiency and safety in air traffic management. The validation took place within the context of the SESAR industrial project, PROSA.
ASR takes audio signals and transforms them into text, for example for visualisation on the controller’s display. The technology integrates artificial intelligence (AI) and machine learning algorithms to consider text predictions based on surveillance data; creating ATM commands based on existing air traffic control concepts; and a combination of both to provide further applications for the controller working position (CWP).
Led by SESAR 3 JU member, DLR, the validation involved 10 air traffic controllers (ATCOs) from Austro Control supported by two safety experts from Integra recently validated the influence of automatic speech recognition on flight efficiency and ATM safety in DLR’s ATMOS simulation environment.
The validation also saw the participation of DLR experts of the ATM simulator and speech recognition and understanding (ASRU), one subject matter expert from Austro Control, one observer from SESAR 3 Joint Undertaking.
The trials were conducted within the context of the PROSA project, which is developing a dedicated ASR solution, and the HAAWAII project which provides expertise ASR architecture and improved transcription and annotation rules.
Each ATCO took part in in four different simulation runs either with a medium or a high (42 arrivals per hour) traffic scenario. Both scenarios were conducted once with ASRU support and once without it. To avoid sequence effects, five ATCOs started with ASRU support while other five started without it (so called baseline runs).
DLR is currently evaluating the effect of ASRU on the flown trajectory length, which correlates of course with kerosene consumption. In baseline runs the ATCOs entered all spoken commands manually into the aircraft radar labels, i.e. they entered heading, speed, flight level and altitude values as well as cleared waypoints, transitions and descent rates. In simulation runs where ASRU was active, these inputs were done automatically, only required correcting actions when ASRU did not recognize a clearance correctly. It can be already stated now, that the number of necessary inputs was reduced by a factor between 8 and 12 with ASRU support.
Left side: Radar Screen with output of ASRU in purple, right side: output of ASRU on word level and on semantic level
The ASRU can also mean enhanced safety since the time saved on inputs can be used to focus on safety-critical tasks. When it becomes necessary, the ATCO can use this additional time for taking care of emergency flights, for coordinating with other sectors or simply for improving the performance by giving the right clearance at the “perfect” time.
These safety buffers were not only claimed, but measured by a secondary task, which the ATCOs performed in addition to controlling the traffic.
ATCO performing secondary task to select the colour of a displayed word
From observations (without detailed evaluation) it can be assumed that the command recognition rate was above 90%. Detailed evaluation of ASRU performance is in progress and will be reported later. Safety experts from Integra, on behalf of air navigation services of the Czech Republic, are currently evaluating, if the remarkable performance of ASRU could also have negative effects on safety due to over trust in the system. The solution in this case is the integration of ASRU output with the downlinked settings from the aircraft, so that an additional safety net is available.
The benefit of the presented ASRU solution: The ATCO can concentrate on his/her main task, monitoring and sequencing aircraft and can avoid the tedious work of maintaining the radar label contents!
More about automatic speech recognition
More about HAAWAII
More about PROSA