Technologies behind immersive VR: positional tracking and VR-accessories. Part 1

A brief history of VR evolution equipment and immersion technologies.

After the explosion of interest in VR in 2016-2017, interest also quickly disappeared. The VR market was disappointing: 5 million active VR devices versus 120 million video games consoles. VR gaming couldn't provide acceptable prices, stability of equipment and quality content. Despite this, the VR market is gradually developing and in 2019 there are new cheaper and more stable devices supporting a rising amount of high-quality VR content.

In this article, we will analyze the development of VR equipment, consider how modern virtual reality helmets work and what additional immersion technologies are on the market.

Positional tracking technologies

The question of determining the position of the object in space is relevant not only for VR, sailors and pilots need to understand where they are and where to keep the course. Engineers working on virtual reality systems are solving a similar problem.

Fun fact, speaking of navigation, at the dawn of virtual reality devices were designed to train military pilots and looked like an evil mushroom.

picture from (https://icdn8.digitaltrends .com/

There are several important characteristics of virtual reality systems:

  • Delay. How quickly the system responds to changes in position.
  • Accuracy. How precisely the system determines coordinates
  • Number of objects. How many objects in space can be determined by the system (helmet, joysticks, additional items)
  • Coverage area. How large is the area available for movement
  • Sensitivity to the environment. For example, how sensitive the system is to external light or to nearby magnetic field distortions.

If the system characteristics perform with a large delay or lack accuracy, what we see does not coincide with the impulses from the vestibular apparatus (you turn the head, but he picture updates with a delay). In many people, this “mismatch” can cause a feeling of nausea.

The nature of this phenomenon is controversial, but there is a common hypothesis based our survival instinct. When in ancient times a person ate a poisonous plant affecting brain activity, they felt dizzy and the body experienced a similar “mismatch” between what it saw and what it felt. To get rid of toxic substances, instinct evoked sense of nausea with subsequent vomiting. The same survival instinct now keeps us from poorly calibrated positioning systems with low accuracy or a large delay.

Let's take a look at the main positioning systems and their features.

Inertial positioning system

The most common device is the inertial measurement unit (gyrostabilizer). IMU consists of several gyroscopes and accelerometers. The gyroscope determines the level of inclination, and the accelerometer – how speed has changed according to the change in coordinates.

picture from wikipedia .org

The gyrostabilizer can't tell where the object is. It can only determine the acceleration by three coordinates. One way to get the path and coordinates is to integrate accelerations twice. But like any other positioning system, such a system gives a random measurement error that becomes quadratic when integrated. IMU-only systems accumulate these errors, which are never reset. The resulting error is called “drift” of inertial systems.

Therefore, virtual reality systems that use only IMU, such as Google Cardboard, give you three out of six degrees of freedom. They only allow you to look around, standing still.

Almost all industrial positioning systems have built-in IMU. It is quite stable and can work without visual input, so when the visual sensor of the system is obscured, the system switches to the IMU. When used for short periods of time, IMU positioning systems show little drift that can be disregarded in favor of overall good results.

To remove the error of integration IMUs are used with another independent positioning system, which constantly corrects the IMU. These systems work simultaneously and the final position is calculated on the basis of data from all of them using Kalman filters.

Acoustic positioning system

In acoustic positioning systems, several sound sources generate an ultrasonic signal, and the microphone system in the room performs trilateration. The position of the object is calculated from the geometry of the microphones.

On the other hand it has some significant disadvantages:

  • The space between the transmitter and receiver should be completely empty not to distort sound waves. That's the reason Acoustic positioning is very rarely used in industrial positioning systems.
  • Low refresh frequence
  • Dependence on the environment. The speed of sound wave propagation depends on the temperature and humidity of the room.

Magnetic positioning system

The magnetic system is the first positioning system that can be used the virtual reality mass market. In such a system there is a source of electromagnetic field with three orthogonally arranged inductors and each monitored body is equipped with a sensor with three coils. The position is determined by the electromotive force induced in the sensor coils.

A similar principle is used in theremin, a musical instrument that produces sounds of different frequency depending on the position of the player's hands relative to the magnetic field around the receiver.

Magnetic systems are quite accurate, but they only operate at a short distance from the source of electromagnetic field. Because of that fall back modern VR barely uses magnetic positioning, while it's very popular in the medical field because of its high accuracy.

Optical positioning system

The most common positioning system at the moment. It works on the same principle as our eyes – the brain receives a stereo image from two sources and creates a three-dimensional image on top of it. Two different image sources allow our brain to assess the position of the object in space.

Take a look at Epipolar geometry and computer stereo vision" to learn more about stereo images.

picture from wikipedia .org

Optical systems are divided into several subgroups: by the presence of markers and by the location of the camera.

Optical positional tracking with markers

Cameras are installed along the perimeter of the room, optical markers are attached to the sensors. Each sensor has its own pattern so that the system can distinguish the sensors from each other.

picture from cnet .com

Sensors can be passive (reflect light) and active (emit light). Active give more stable tracking and more accurate position determination.

Optical positional tracking without markers

In this case, instead of markers with fixed patterns, the system itself tries to find “anchors” on the image. The system determines the selection points (angles and faces) and then correlates the change in position of these points with the IMU data. One of these algorithms we will consider later.

picture from venturebeat .com

Outside-in tracking in VR

Let's move on to the second parameter that categorises optical systems – the position of the camera. In Outside-in systems, objects are tracked with a camera fixed at a stationary location. As long as the objects are within the range of the camera, the positioning works quite well. The classic representative of such a system is Playstation VR.

picture from playstation .com

Inside-out tracking in VR

In this case, the cameras are installed on the VR-devices. Inside-out devices then gives you a considerably more freedom to move compared to Outside-in devices. Both leading VR headsets, HTC Cosmos and Oculus Quest, mostly use this positional tracking system.

pictures from hi-fi.ru and pisces.bbystatic .com

In Part 2 I'll cover devices that provide immersive experience including headsets, treadmills, and VR apparel. Stay tuned!

Read similar articles

Blog = AR Technology: How Does Augmented Reality Work?

AR Technology: How Does Augmented Reality Work?

Also where it's used and how it's differences from MR and VR.

Learn more 6 min read
Blog = How to Develop an iOS App —  A Step By Step Guide For Beginners

How to Develop an iOS App — A Step By Step Guide For Beginners

A plain-language description of the essential steps for creating an app for iOS.

Learn more 7 min read
Blog = Why Digital Identity is Going to be Blockchain’s Killer Dapp

Why Digital Identity is Going to be Blockchain’s Killer Dapp

It's immutable, it cannot be tampered with, and it has levels of permissioned viewing.

Learn more 10 min read