How does virtual reality work?

What is VR?

Virtual reality (VR) is where computer technology creates a simulated environment that places the user inside an experience. The user isn't restricted to viewing things on a screen, but instead can interact with immersive 3D worlds. These worlds do not actually exist, which is why they are virtual. However, the technology that drives VR can create experiences that feel immersive and believable. The world the computer creates is one you can explore, and one which stimulates your senses. The user feels that they are present in the virtual reality world, both physically and mentally. As a concept, VR is not a recent phenomenon. The aim of creating something that the viewer, or user, finds immersive dates back some 200 years. For example, in the 18^th and 19^th centuries, artists created paintings of vast, panoramic views with the aim of putting the viewer right at the heart of an immersive scene. In Victorian London, the stereoscope became hugely popular. This device combined twin images to create three-dimensional virtual worlds. Modern VR is more sophisticated and is based on key technological facets.

How Does Virtual Reality Work?

The technology that makes modern VR possible is primarily about simulating human vision. To do this, it must trick the brain into thinking the virtual world is behaving like the real one it is simulating. The brain must process sensory information and then interpret it. This interpretation is our perception of the world around us. The interplay of various sensations makes up our perception. The task of VR technology, therefore, is to influence our perception of what we are experiencing. It does this using various elements, including:

• Stereoscopic vision
• Head tracking
• Eye tracking
• Motion tracking

The vehicles for combining these technological elements into a user experience are:

• VR headset
• VR projection

Stereoscopic vision

We have already mentioned the stereoscope as an early example of a device that creates an immersive experience for the viewer. Stereoscopic vision is a key element in VR technology. For someone to experience an image as three-dimensional, they must have binocular vision. The left and right eye see things from slightly different viewpoints. The brain merges these two strands of information together in a process known as stereopsis. We see in stereo. To reproduce this effect for VR, each eye needs to have the image presented to it differently - something a headset does. VR projection uses multi-projector technology to create large-screen stereoscopic experiences.  

Head tracking

Motion is a big part of VR experience. If a VR headset simply displayed the same stereoscopic image no matter whether we looked up or down, or side to side, then it would quickly feel unconvincing. Head tracking in one of the major ways that VR can simulate what the user experiences in real life, in effect fooling the brain. Head tracking sensors monitor the direction a viewer’s head is pointed in, detecting movement and direction of movement. The system plots the head on an XYZ plane, and measures head movements. Depending on the sophistication of the specific VR device, head-tracking responsiveness will vary. High-end headsets use precise sensors for accurate tracking, including cameras, infrared LEDs and magnetometers. More commonplace devices, such as smartphones, depend on accelerators and gyroscopes to track head movement. See below for more about motion sensors.

Eye tracking

Eye tracking works with the subconscious reactions and behaviours of the individual experiencing the VR world. This is a more advanced form of tracking, and it works by a continuous measurement of the distance between the centre of a pupil and the reflection of the cornea. Invisible to the human eye, an infrared light creates the reflection and built-in cameras record and track movements. Complex computer algorithms can work out where the human gaze is directed from the recorded angle of the eyes. Ultimately, eye tracking can make VR more efficient, since the equipment only requires the processing power to render those elements of the virtual environment that the eye is looking at. It also makes the virtual world represent the real one more closely, creating a more immersive environment.

Motion tracking

For a VR experience to feel immersive, the viewer has to experience their own movement in the virtual world as being as close to reality as possible. Without motion, the virtual self becomes paralysed, and the immersive effect is diminished or lost. The premium immersive experience puts the whole body into the VR experience. Obviously, in some virtual situations, such as driving a vehicle or piloting an aircraft, full body motion is not necessary, but the experience has to capture a sufficient range of motion to keep things feeling real. Motion tracking is based on the principle of six degrees of freedom, or 6DOF, which refers to how a rigid body moves freely in three-dimensional space. It has two broad categories:

• Optical tracking
• Non-optical tracking

Optical tracking relies on imaging devices to track body motion. This method generally uses cameras and optical markers on the person being tracked. These markers may be passive, which reflect light, or active, which are computer-controlled LEDs. Generally, active optical tracking is not widely used in consumer VR products. There are also non-marker based systems for optical tracking, which combine infrared and depth-sensing cameras; or use real-time scans of objects in close proximity to the body. Non-optical motion tracking methods use accelerometers, gyroscopes and magnetometers. These microscopic devices measure the various orientations of whatever they are attached to:

• Accelerometers measure movement along the XYZ axes
• Gyroscopes measure 360° rotation
• Magnetometers determine orientation towards a magnetic field

Various VR devices and systems combine non-optical tracking methods with infrared tracking or passive reflectors for highly accurate motion tracking. Achieving low latency Effective VR depends on achieving low latency. Latency is the delay between an action and a reaction. So, if you move your head physically in a virtual world, you need to experience the changes that result immediately. Otherwise, this will cause your brain to reject the legitimacy of the virtual scenario you are in, effectively breaking the spell of perceiving you are experiencing something real. End-to-end latency is the delay between the movement of the user’s head and the change of display in the VR device. The challenge for any VR system is to minimise latency. The greater the latency, the less realistic the VR world will appear. The human brain can detect tiny amounts of latency in VR systems. Therefore, the aim is to bring latency down to a level that is undetectable by the human brain. However, too-high latency can cause simulator sickness, which is a form of motion sickness experienced in VR environments. Along with precision tracking, the faster the refresh rate of the VR device, the more realistic and fluid the user’s experience will be. The refresh rate is how many frames per second the device shows images. Download our ebook on VR

The future of VR

Rapid advancements in technology mean that truly immersive digital experiences are coming. Already, the capabilities of VR have seen this technology expand well beyond the boundaries of gaming and other forms of interactive entertainment. VR has applications in the design, retail, engineering, manufacturing, medical and military sectors. It is fast becoming an indispensable tool in areas such as building design and urban planning, and in training and assessment for an increasingly wide range of skills. Virtual reality provides the platform for sophisticated data visualisation, immersive marketing techniques and holistic training. Soldiers can familiarise themselves with equipment in risk-free environments. Surgeons can learn and practice virtual skills before transferring them to the live operating theatre. Therapy programmes for treating mental illness and PTSD could benefit from adopting and integrating VR technology. VR is a rapidly growing technology sector.

• Research indicates that the global market size for both VR and AR is 18.8bn US dollars
• The estimated forecast for global shipments of VR head-mounted displays in 2024 is 25.3 million

From a real-world view, the future of VR looks limitless.

The Bottomline

VR systems can provide dedicated support and functions across different disciplines and industries. At Antycip, we provide tailored VR solutions and immersive virtual environments:

• VR CAVE (Cave Automatic Virtual Environment) solutions for immersive training and research and review
• VR collaboration in 3D and real time, across a broad range of applications, improving communication in shared virtual workspaces
• VR design and review, including virtual prototyping and collaborative design reviewing

For more information about VR, its many applications, and how this technology can support your work, contact us.