Media Technology MSc

Multimodal Augmented Reality: The Norm Rather Than the Exception


Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case for an experience-focused and modalitiesencompassing understanding of AR. We argue that multimodality in AR is the norm rather than the exception, as AR environments consist of both virtual content and our real, physical, multimodal world. We explore the role multimodal and non-visual aspects of our physical reality can play when creating AR scenarios and the possibilities and challenges that emerge when approaching AR from a modalitiesencompassing perspective.


augmented reality, multimodality; interaction; multimodal interaction, multimodal augmented reality; non-visual augmented reality; mixed reality; theory; experience; perception; virtual; real; senses




Hanna Schraffenberger and Edwin van der Heide, Multimodal Augmented Reality — The Norm Rather Than the Exception. In: Proceedings of the ACM Workshop on Multimodal Virtual and Augmented Reality, 2016