Augmented Reality as an Aid and a Distractor: Pilot Behavior in Normal and Abnormal Flight Scenarios

Abstract

In the information-dense and time-compressed cockpit environment, augmented reality (AR) displays are a promising approach to provide pilots with supplementary information. This research investigated head-worn, optical see-through AR systems, which project digital images while providing a clear view of the real world. While the benefits of having helpful information readily accessible are clear, AR displays bring several unique concerns from a human factors perspective. This research pursues a more complete understanding of both favorable and adverse impacts of AR displays on pilot performance through two user studies. The first study investigated the effects of AR display occlusion of the real world on pilot performance, assessing the impact of AR graphic presence and location. The second study investigated the influence of AR on pilot objective flight performance, workload, situational awareness, and usage when serving as either an aid or a distractor for an aircraft landing task. The findings from both studies reflect the feasibility for AR display usage in the cockpit, as well as the importance of understanding the potential for adverse performance, trust, or cognitive impacts from its use. This work contributes a novel investigation into the drawbacks of AR displays in the aviation domain, utilizing a multi-modal approach to understand both flight performance and cognitive impacts of AR systems. Further research is needed to understand the complete picture of AR use in the cockpit and ensure that AR displays support pilot safety, even in the case of malfunction or failure.