Manipulating vision for immersion
In this article I’ll start to explore some of the technologies that are being used to achieve immersion.
Our bodies are constantly interpreting patterns of sensory input to make sense, or perceive the world around us. Our senses and perception are what immerse us in reality.
In order to create a virtual reality, we need to control the perception of our senses- vision, hearing, touch, smell, and taste. This can be achieved by controlling sensory input. The ultimate virtual reality could be achieved if one has complete control of the inputs to all senses. Over time various techniques and technologies have been used to manipulate or generate inputs for each of these areas.
The point at which a user feels highly immersed is now quite achievable with commercial technologies. Now I’ll take a look at vision, with later articles looking at some of the other senses.
Vision
Stereoscopy
Stereoscopy is the use of two dimensional images to create an illusion of three dimensional depth for humans who have binocular vision.
Our brains combine two images using our eyes which are located a short distance apart. Our brains use the difference between these along with the size of objects to interpret the depth of a scene.
In stereoscopy two images of the same scene are captured from a distance apart, and each of these is displayed one to each eye of a viewer using one method or another, thus the illusion is created. This works for both still images which were first popularised after the 1851 Great Exhibition, and for video. The technique proved popular for both entertainment and professional uses.
The technologies by which these separate images or image streams can be presented to a viewer’s eyes have varied over time. Stereoscopes such as the Holmes model pictured below use one lens for each eye to help unite the two images, allowing for a reduction in size of the device which enables a more portable design.
Virtual Reality Headsets
Virtual reality headsets build on the stereoscope by accepting positional and rotational head movements and changing the images accordingly. When the system is responsive enough this leads to the illusion of the viewer feeling immersed within the virtual scene.
Looking at the Holmes stereoscope, and comparing it to a present day head-mounted display (HMD) like the Oculus Rift (pictured below), there are obvious similarities. Both feature one lens per eye, plus a method for displaying separate images to each eye. There is also an ergonomic hood to block ambient light. The main difference is the source of the image- the Holmes stereoscope using two static analogue photographic prints, whereas the Oculus Rift uses an LCD screen capable of still or video playback at 80Hz refresh rate.
As with other digital formats the resolution of the displayed data determines the fidelity of the media, in this case a series of images. The first consumer VR HMDs became available in 2016, and since then screen resolution has been increasing as new models are introduced, as can be seen in the consumer headset comparison image below.
Field of view (FOV)
Field of view defines the angle in degrees that one can see, and is an important factor contributing to level of immersion. The greater the FOV, the larger the scene.
The average human field of view is 180° horizontally and 130° vertically, so these make obvious desirable targets for HMD specifications. Binocular vision covers 120° and while it seems logical to design for this area, peripheral vision also plays an important part in one’s observation of an environment, thus headsets with less than 110° view angle generate a tunnel effect. (Arnaldi, Guitton & Moreau 2018).
Augmented Reality Glasses
While VR affords the user complete immersion, AR glasses offer a way to overlay what you are seeing with digital information, making mixed reality a possibility. With this comes the ability, with ample sensor data and computation power, to detect the real world and have digital experiences interact with it as though it is in the same space. This can produce a highly immersive experience. Here is an image of Nreal Light AR glasses which are being made available in the EU some time during early 2021.
Currently AR glasses have a smaller field of view than VR headsets, limiting the potential for content creators and thus limiting immersion. This is on track to increase over time and I foresee a time when we will have the option of full field of view AR.
Summary
The control of vision is the best it has ever been. HMDs (including AR glasses) are coming down in price and increasing in capabilities, this coupled with increased computing power will, over time, make the manipulation of vision easier with more lifelike results. HMDs will become more comfortable enabling longer in-headset sessions, opening up new possibilities for content creators. As HMDs become more ubiquitous the increasing size of the market will fuel further content creation and advances in hardware and software capabilities and applications. These advances will further enhance the effect of immersion.
Artists and creators can take advantages of the latest developments in vision control making it easier to achieve more lifelike experiences- resulting in greater levels of immersion. This is important for immersive storytelling because it makes it easier for the audience to achieve suspension of disbelief, leading to deeper and more resonant experiences with more people.
In my next article in this series I look at developments in the control of hearing.
References
Arnaldi, Guitton & Moreau 2018. Virtual Reality and Augmented Reality: Myths and Realities. Bruno Arnaldi (Editor), Pascal Guitton (Editor), Guillaume Moreau (Editor). ISBN: 978-1-786-30105-5 March 2018 Wiley-ISTE