Building on the results, the ASRU tool was progressed within the PROSA (PJ.10) industrial research project, and designed to not only recognise voice commands, but more importantly, understand them. “Once you know the context, it becomes easier to understand the meaning,” explains Prof. Dr. Hartmut Helmke from the DLR Institute of Flight Guidance. “Speech recognition can identify the individual words, but understanding them is key.” DLR developed a speech understanding system using tens of thousands of controller-pilot transmissions supplied by PROSA partner, Austro Control, and HAAWAII project partners, Isavia ANS and NATS. Validations trials were performed with twelve controllers from Vienna approach control in DLR premises in Braunschweig. Controllers only needed to check radar labels and input corrections manually if there were errors. Based on about 120,000 spoken words, 9,000 transmissions and 18,000 commands, the validation recorded 3% word error rate, command recognition rate of 92% and call sign recognition rate of 97%. During 28 hours of simulations, mouse-clicking time fell by a factor of more than 30, from 12,800 to 400 seconds. An earlier project, known as AcListant®-Strips had already shown that in the busy terminal environment, the reduced workload enabled controllers to deliver more timely instructions, estimated to save on average 77 seconds flight time per aircraft, reducing fuel consumption by 60 litres.
The PROSA project evaluated for the first time, how many errors of ASRU are not corrected by the ATCo. They found that ASRU was capable of correctly recognising 92% of controller commands. That means that the tool incorrectly recognised 8% of the commands, half of which controllers were able to correct manually. This may appear on the surface to represent a safety risk. However, DLR and Austro Control evaluated for the first time the number of wrong and missing radar label entries, when ASRU support is not available, i.e., they compared these results with current air traffic control operations. They found that 11% of the given commands were missing or wrong, which is much more than the 4% with ASRU support.
Prof. Helmke and his team are now ready to test readback error detection support, another ASRU application enable by the HAAWAII architecture, in an operational environment to find out more about readback errors and what ATCos really need to reduce the number of undetected readback errors. The tool is simple to use with green, yellow and red alerts appearing instantly in the flight label informing controllers when errors need to be corrected. “We can also use the tool to collect more granular data, for example to show different aircraft types and airlines and carry out further analysis.” Identifying what constitutes a readback error and which ones should be brought to the attention of the controller presents the more challenging area of research where “a rule-based system has the potential to be very successful,” he adds.
DLR is taking the research further with the addition of a “digital co-controller” that acts as a digital check on controller actions. The co-controller can be trained to recognise message inconsistencies and can draw attention to events or issue prompts based on neural network learning. The aim is to distribute responsibility between the controller and the digital co-controller, so both contribute to safe traffic management and deliver improved performance for airspace users. Installed in a control tower for example, ASRU can be combined with ground tools to alert when an aircraft takes a wrong turn, or to identify optimum routings.
DLR’s ASRU tool has reached technology readiness level 6 (TRL6) and is ready for industrialisation where it could lead to more effective use of controllers’ skills, says Prof. Helmke.
More about MALORCA, HAAWAII and PROSA.
More about automatic speech recognition
