Technologies behind immersive VR: positional tracking and VR-accessories. Part 2

A brief history of VR evolution equipment and immersion technologies.

VR Headsets for gaming

In Part 1 we studied positional tracking technology development, and here let's take a closer look at VR-headsets and other hardware that makes VR immersive.

Oculus Rift

photo from http://www.openhmd. net

Oculus Rift uses the Outside-in positioning method called “constellation.” The headset and controllers have infrared markers invisible to the bare eye. The infrared camera is installed in front of the play area and determines the position of the markers. The resulting image is rendered in the headset for the player to see.

Also, the headset and controllers have an IMU installed. The final position of the devices is calculated according to the data from the IMU and the optical sensors.

HTC Vive

In the Lighthouse positional tracking system, sensors are placed inside the headset, in contrast to Rift, where the sensor calculating the position is outside.

The play area is surrounded by base stations. HTC Vive headset has a photodiode system that detects infrared radiation emitted from the base stations. Inside the base stations, there are two stepper motors that spin two infrared lasers at a constant speed with a Fresnel lens scattering the laser into a plane. The algorithm is as follows:

  • Base station #1 produces a big flash and it signals the other base station to stop working. All photodiodes reset the timer and start the countdown.
  • When the horizontal laser plane crosses the photodiode, the photodiode remembers the time. Given that the speed of the motor is preset and constant, the device can calculate the angle between the photodiode and the station.
  • Then the procedure repeats for the vertical plane.
  • The calculated angles of each photodiode are converted to the position of an imaginary camera, and for that group of photodiodes a PnP problem is solved.

We used the Lighthouse system for one of our projects, and as it turned out, there are a lot of nuances that worsen the stability of Vive's tracking system:

  • Direct sunlight entering the play area. We had to work in constant shadows with blinds down.
  • Presence of reflective surfaces in the play area (mirrors, glass).
  • Sensitivity electromagnetic noise at a frequency of 2.4 GHz. That kind of interference can be a result of a large number of WiFi routers working around the play area or a broken microwave being nearby. If you are thinking of opening a VR Arcade with HTC Vive, don't do it in a shopping mall or anywhere where many people are likely to use WiFi.
  • The instability of SteamVR platform. Lighthouse needs SteamVR software for proper functioning, but the platform itself doesn't always work properly. Usually, to solve positioning problems, you need to turn SteamVR off and on again.

Oculus Quest

Quest, similar to HTC Cosmos, use Inside-out optical positioning systems. No additional sensors or stations are required around the play area.

Oculus Quest looks magical from the outside. You put your helmet on, and it starts working:

  • You don't need to prepare the room.
  • You don't need to have a computer connected.
  • Hence there are no wires (I thought this, although being evident from the above, merits it's proper bullet).
  • There are almost no restrictions on the area of playing space (I walked out from our office on the sixth-floor to a nearby cafe and Oculus still could track the starting point).
  • The cost of the device is $400 (compared to almost $3000 total cost of HTC Vive+MSI VR One powering it).

On the downside, Oculus Quest has less powerful hardware if compared to a PC so that AAA games won't run. However, as a user, I am thrilled with Oculus Quest.

Technologies behind free roam games

In VR games, there are a few movement mechanics:

  • The player can move, and the distance covered by their avatar in VR equals the distance the player covers in the play area.
  • The player stands still and can teleport their avatar in VR.
  • The player stands still and can move their avatar backward and forward with controller buttons. The latter option greatly reduces the level of immersion in virtual reality. There are though several technologies to address this issue.

Kat VR Treadmill

A Chinese-American company that develops treadmills for virtual reality. You put on special shoes, stand on the treadmill; the treadmill tracks your movements and transfers data to your avatar. Kat VR Mini costs around $5000.

The company is now raising money on Kickstarter for its new product “KAT Loco”. Ankle sensors detect jumping from one foot to another, and that counts as walking.

Optitrack motion capture solution

Optitrack is the leader of free roam solutions on the market used by major VR arcade franchises, like SandboxVR and TheVoid, use Optitrack.

Optitrack cameras are installed throughout the play area to track both movable and immovable objects. Optitrack produces its own cameras to insure a high update frequency and high resolution.

picture from https://optitrack .com/

The biggest disadvantage is the cost. An area of 30 by 30 meters for 48 objects (12 people) costs $500.000 as shown in the screenshot of price calculator from the Optitrack website.

picture from https://optitrack .com/

Simultaneous localization and mapping

An alternative to Optitrack and other Outside-in positioning systems is Inside Out systems. Its main representative is SLAM (Simultaneous localisation and mapping) algorithm that is already used in the latest HTC Vive Cosmos and Oculus Quest. The camera attached to the headset analyzes the environment and builds a map of the room. The controllers work on the IMU and the headset cameras determine the position of the controllers by movement patterns.

The greatest thing about SLAM is the ability to use any room for VR and almost no space restrictions. On the other hand, there is no support for additional items, as there are no static cameras and sensors that can track them.

What makes the most immersive VR games

VR used to copy videogames realities where the player was an incorporeal creature lucky to have hands (sometimes hands also were no option). It didn't influence the gameplay; it just didn't seem very real. Now many VR games and applications allow users to see their full-bodied VR avatar and interact with physical objects.

VR weapons

HTC has trackers that can be attached to weapons, like baseball bats, guns, and torches. Feeling the weight and momentum of a bat when you swing it gets immersion to the next level.

photo from https://www.gomgi .com

Full body tracking in VR

The kinematic model of a person allows you to see the whole body in VR. Typically, you need sensors are attached to every major joint to create a model, but a VR avatar can also be built with only 5 sensors attached to hands, feet and head. Of course, the more sensors you get the greater the accuracy is.

Haptic gaming vests and suits

Headsets provide input for two of our senses, sight and hearing, and while taste and smell are still off-limits, there is a way to input touch. With a vibration vest on you can feel the blows drawn at you. You might have already heard about the bHaptics company and their devices.

picture from https://www.bhaptics .com//

In addition to the vest, they develop sleeves for hands, arms, and feet and cushions for the face.

Another company is TeslaSuit. Judging by the description, in addition to the feeling of touch, the suit will allow you to feel warmth and cold. The suit is not available even for pre-order yet though.

picture from https://teslasuit .io/

VR hand gloves

Leap Motion used to develop an infrared sensor that can be attached to any VR headset, which would not only track the position of your hands but the position of your fingers too. There was one restriction, that sort of robbed LeapMotion of its coolness; tracking would only work if you don't have anything in your hands. Also, any non-standard hand positions were tracked extremely poorly and you couldn't use any controllers.

Last year the company was rumored to be sold to Apple, but in the end, the deal did not take place. In May of 2019, LeapMotion was bought by UltraHaptics for $30 million. At the peak of popularity in 2013, Leapmotion estimated at 300 million.

The Haptxgloves company develops VR gloves that allow you to feel the weight and volume of objects. So far, they look rather shabby. Early Access is available on the Haptxgloves website.

Summary

The virtual reality market continues to develop slowly. There are new cheaper and more stable virtual reality systems, and additional equipment allows you to dive into VR even more.

Recently we've got my hands on Oculus Quest so that a review will follow soon. Stay tuned!

Read similar articles

Blog = Technologies behind immersive VR: positional tracking and VR-accessories. Part 1

Technologies behind immersive VR: positional tracking and VR-accessories. Part 1

A brief history of VR evolution equipment and immersion technologies.

Learn more 5 min read
Blog = Top 36 Mobile App Development Software in 2019

Top 36 Mobile App Development Software in 2019

Compare features, prices, and ratings about each tool to choose what fits you.

Learn more 11 min read
Blog = Why Digital Identity is Going to be Blockchain’s Killer Dapp

Why Digital Identity is Going to be Blockchain’s Killer Dapp

It's immutable, it cannot be tampered with, and it has levels of permissioned viewing.

Learn more 10 min read