Automation is a valuable part of complex and labor-intensive industries like air traffic control, but how personalized and transparent should digital information be for air traffic controllers? What trade-offs exist, in terms of controller trust, acceptance and performance? 

To answer these questions, MAHALO developed a hybrid machine learning (ML) system composed of layered deep learning and reinforcement learning models, trained on controller performance, strategies, and physiological data.  It learned to solve ATC conflicts and, coupled with enhanced en-route traffic conflict detection and resolution (CD&R) prototype display, it presented machine rationales for the ML output. The ML system was then tested with Italian and Swedish air traffic controllers to evaluate in real-time simulations the relative impact of ML conformance, transparency, and traffic complexity on controller understanding, trust, acceptance, workload, and performance, to guide the design of future AI systems.

MAHALO sought to make the AI agents fully cooperative with the human controller rather than just a “black box”  that outputs solutions when necessary. It used both a  novel approach to interfaces in ATM (such as solution space diagram metrics as input an o ML agent) and a mixture of conformal and optimal automation to test the best combination and acceptance uptake among controllers. In that way, MAHALO considered both accuracies of the automation and controller acceptance.

The research found a statistically significant effect of conformance agreement, and also on reduced controller workload ratings. Moreover, even if there was no statistical significance, data trends suggested an interaction between conformance and transparency. It provided clear guidance on how ML should be developed to keep the human “in-the-loop”, and for artificial intelligence agents to act as team players that understand the user’s needs, not to be just a machine giving out outputs when asked.

Benefits

• Reduction in perceived workload
• Maintains the human-in-the-loop
• Guidance for future research