Haptics in Music: Exploring the role of audio-tactile vibrations in experiences of virtual reality media.

Proposed by Gareth W. Young
Email: YoungGa(at)tcd.ie

Sound and touch are naturally linked and can be explored together in human-computer interactions that involve new digital media. As the auditory manifestation of sound, vibrations can reach listeners through the air. Therefore, we listen to music not only with our ears but with our whole body too. This simple act of acoustics provides multi-modal stimulation to the listener, as a performer (active) and the audience (passive) alike. A complex system of auditory–haptic interactions has been established between music, musicians, and audiences that can render subtle differences in the experiences of musical performance. At a live concert performance level, auditory–haptic interactions can provide insight into how we create rewarding experiences for entertainment via new media platforms, such as cross-reality (XR) devices like augmented reality (AR) and virtual reality (VR). This research project, therefore, aims to explore the relationship between audio-induced vibrations on the body and musical experiences delivered via XR. By controlling sound and tactile vibration separately, psychophysiological experiments will be performed to investigate their individual and combined role in the perception of and experiences of both active and passive encounters with digital musical media and XR specific performances. This research will build knowledge of the psychophysical systems that can be implemented to generate both music and vibration for AR and VR technology. 

Bibliography:

Young, G. W., Murphy, D. and Weeter, J. (2018). “A Functional Analysis of Haptic Feedback in Digital Musical Instrument Interactions”. Chapter 6 in Musical Haptics, Springer Series on Touch and Haptic Systems, by S. Papetti and C. Saitis (eds.), Cham: Springer, pp. 93 – 122. 

Young, G. W., Murphy, D. and Weeter, J. (2017). “Haptics in Music: The Effects of Vibrotactile Stimulus in Low Frequency Auditory Difference Detection Tasks”. IEEE Transactions on Haptics, 10(1), pp. 135 – 139.

Young, G. W., Murphy, D. and Weeter, J. (2017). “A Qualitative Analysis of Haptic Feedback in Music Focused Exercises”. New Interfaces for Musical Expression, Aalborg University, Denmark, pp. 204 – 209.