Bridging the Blue

---

Creative Experiments with Volumetric Video in Virtual Reality.

We present Bridging the Blue an immersive creative experiment that explores virtual reality (VR) as “the ultimate empathy machine” where users can explore an imaginary world and experience personal representations of clinical depression. In this creative experiment, we explore the role of immersive content creation in technologically mediated perspective-taking experiences of VR. Bridging the Blue harnesses VR as a new type of human-orientated technology that can be used for perspective-taking, by applying the volumetric capture of real-world human performance and content processing to assess the effects of empathy-building experiences that can be achieved via new media. Audience attitudes towards a protagonist can be augmented to re-evaluate their experiences of perceived oneness with others and increase the amount of effort spent on helping oneself, others, and meaningfully engaging with society generally. From an industry perspective, this type of technology intervention can be used to discuss production methodologies and interaction techniques when applied in mixed reality content creation and to critically inform future creative technology projects presented in VR.

Bridging the Blue is an interactive VR experience that transports the viewer to a familiar, yet imaginary world. Captured as a live studio performance, philosophies of augmented virtuality are applied through the live-action capture and representation of the performer who is projected into and interacts with the virtual world; that is, objects from the physical world co-exist with digital objects within the immersive virtual environment (IVE). Through the merging of the real – the volumetric capture of the narrator – and the virtual – the world presented within the IVE – Bridging the Blue produces a new experience of VR for users to explore the experiences of depression. Within each scene, the work uses different narrative disclosure techniques to challenge common misconceptions of and responses to depression and offers subjective playback and validation to those who have also been affected by this condition. The viewer is either addressed directly by the protagonist or occupies a fly-on-the-wall perspective over several different scenarios. The use of VR in this context arbitrates the innate capabilities of the medium, the literary technique of defamiliarization, and the capacity of the medium to inform the viewer.

Science, media, and technology converge in this innovative creative experiment to explore the impact of new mixed reality technologies on issues of social importance that can be enhanced via technologies with human-centered designs. VR presents its users with IVEs via multimodal sensory stimulation – visual, aural, haptic – allowing them to directly experience a variety of interactive, immersive, and imaginary experiences. This technology, therefore, presents value to new media industries in creative applications that serve to create technology-mediated experiences that can be explored to facilitate novel and innovative ways to use new technology for promoting empathy and perspective-taking. Where traditional perspective-taking exercises rely on the participant to imagine a self-other merging process happening, VR can allow an individual to virtually embody another via first, second, and third-person narratives delivered via technology-mediated experiences. This enables viewers to easily and effectively portal into the life of somebody else, where they can potentially see, hear, and feel through another’s body and control their point of view in real-time.

Artist Statement — Lubna Gem Arielle

Bridging the Blue is a Virtual Reality work inviting the viewer into the artist’s mind via a virtual world containing personal iconography derived from the artist’s lived experience of severe clinical depression. It seeks to challenge common misconceptions of and responses to depression as well as offer subjective playback and validation to those who have also been ill in an attempt to reduce shame, stigma and silence.

The artist is encountered as a vologram (volumetric video captured from the real world) relaying experiences of being unwell using non-linear storytelling and commissioned music. The work investigates the reputation of VR as an ‘empathy-making machine’ to close the empathy gap stemming from a tendency to tell or advise rather than listen non-judgmentally as a response to someone being mentally unwell mediated by the novelty-factor of VR as a medium, what Schlovsky might term ‘making strange.’ It asks whether this potential to change the quality of attention can be harnessed as a device to shift perceptions.

The work explores the synergy between humans and technology offering a virtual gateway into empathy that is often absent in the real world through an embodied encounter with the artist as a vologram within a virtual environment. Technology acts as an enabler in creating a virtual world from the detritus of the artist’s mind in which one has agency to move naturally in each scene and to choose the order of episodes At the same time, the use of technology contracts one’s agency as choices to interact with the artist/vologram are limited to relative positioning, where and whether to look or listen but cannot extend to advising or telling – inadvertently modelling responses more consistent with empathy and support in the context of non-professional conversations about mental health. As such, what might be regarded as the coldness of technology induces a rebirth of human warmth.

The work seeks to challenge norms of interactive storytelling by using the virtual experience as a device to access and expose the mental and emotional space of lived experience, not only in the immediacy of VR as a medium, but also in its execution and process. The non-linear storytelling echoes the distortions of time and memory experienced in breakdown. Underlying that, the technological process in creating the vologram, of breaking down streams of video footage from multiple angles, reconstructing these first into a mesh-only depiction of the artist before adding texture tracked the experience of breakdown and recovery, including working with a counsellor to reactivate life. Each involved reconstruction of the artist albeit on different timelines and realities.

Production Credits:

Creative Director: Lubna Gem Arielle
Written & Performed by Lubna Gem Arielle
Volumetric Video Production & 3D Design: Matthew Moynihan, Iman Zolanvari, Rogerio Da Silva, and Alan Cummins at V-SENSE, Trinity College Dublin
Music composed by Evangelia Rigaki
Music performed by Parabasis (Percussion: Richard O’Donnell; Cello: Martin Johnson)
Audio Recording Engineer: Conall O’Maolan
Scenography: Neill O’Dwyer
Producer: Aljosa Smolic

Technical Requirements for Performance:

• Floor space of 4×4 meters
• Small table and chair
• State-of-the-art VR headset with integrated headphones such as HTC Vive (preferred) or Oculus Rift, with the corresponding tracking system
• Powerful VR-ready PC with Internet access
• Power sockets and extension cords

Performance Description:

This is a VR work that is viewed solo through a VR headset in a 4×4 meter physical space of which patrons can safely move within. The virtual world is constructed from the artist’s representation of breakdown (clinical depression) and patrons meet the artist in the form of a volumetric video, also known as a vologram. The non-linear storytelling experience is formed via seven unique episodes. The viewer starts on an island that contains seven icons and artifacts on rocks, which glow green when the VR system handset is pointed at them. The patron can then select an icon to trigger an episode of the story. The stories take place within two separate environments, with the artist narrating each episode accompanied with originally composed music in the background. Each episode is repeated until the patron chooses to leave and return to the island. Returning to the island is triggered by selecting a shadow of a rose within the scene. Transitions back to the island fade to black, giving the audience time to reflect upon the scene they have just experienced. Each scene can be revisited if the patron so desires. It takes between 10 to 15 minutes to experience the whole story, and previously this has been managed through bookable time slots.

Outputs:

Arielle, L. G. and Smolic, A. “Bridging the Blue“, in The Art Exhibit at ICIDS 2019 Art Book: The Expression of Emotion in Humans and Technology, edited by Ryan Brown and Brian Salisbury, pp. 15-27, Carnegie Mellon University, Pittsburgh: ETC Press, 2020, ISBN: 9781716510809.

Arielle, L. G. (Creative Dir.) and Smolic, A. (Producer). Bridging the Blue [Interactive VR Experience]. 19 – 22 November 2019, the International Conference on Interactive Digital Storytelling (ICIDS), Salt Lake City, USA.

Arielle, L. G. (Creative Dir.) and Smolic, A. (Producer). Bridging the Blue [Interactive VR Experience]. 19 – 20 June 2019, Open Quarter, Folkestone, UK.