Exploring Passthrough, Voice, and User Factors Beyond Interaction Modalities in Extended Reality
Kojić, T., Vergari, M., Warsinke, M., Ali, D., Möller, S. & Voigt-Antons, J.-N.
Published in ACM Transactions on Multimedia Computing, Communications and Applications , 2026 — Just Accepted, online 7 November 2026
Abstract
With the increasing adoption of Extended Reality (XR) across diverse application domains, understanding how interaction techniques and individual user characteristics jointly shape user experience has become increasingly important. This paper investigates how combinations of hand tracking, voice input, and passthrough technology influence user experience in XR while examining how individual user characteristics moderate these effects. A controlled within-subject user study was conducted using standardized questionnaires (IPQ, SAM, and UEQ-S) together with objective behavioral measures. Results indicate that multimodal interaction significantly influenced both hedonic and pragmatic aspects of user experience, with the combination of passthrough and voice input emerging as the preferred interaction configuration. Furthermore, prior VR experience was identified as the strongest moderator of presence, emotional responses, and perceived interaction quality. Within the investigated XR task, passthrough was generally preferred over non-passthrough conditions, while voice input improved usability, although passthrough could introduce visual distractions depending on task context and user characteristics. These findings contribute to the design of adaptive multimodal XR interfaces by demonstrating the importance of considering both interaction techniques and individual user characteristics when designing immersive experiences.
Cite as
Kojić, T., Vergari, M., Warsinke, M., Ali, D., Möller, S. & Voigt-Antons, J.-N. (2026). Exploring Passthrough, Voice, and User Factors Beyond Interaction Modalities in Extended Reality. ACM Transactions on Multimedia Computing, Communications and Applications. Just Accepted. https://doi.org/10.1145/3861338
