The benefits of face tracking for autostereoscopic displays

• 10 May, 2020
• by Valentina Zezelj

Autostereoscopic displays deliver 3D effects without requiring the viewer to wear any special devices such as shutter glasses. For them to work, the eyes must be in the optimal position, which limits the viewer’s freedom of movement. To make sure viewers can enjoy the 3D experience without such restrictions, face tracking can be used.

Face tracking for autostereoscopic displays

The illusion of depth in an image is achieved by using special screen-based technologies, inspired by our own visual system. Human brain extracts the depth information by fusing together two separate images acquired by our eyes. Similarly, autostereoscopic displays present a different image to each eye.

The most popular way to achieve that is the parallax barrier. It’s a series of transparent strips that allow each eye to see different regions of the display screen. The “right eye” image is sent to the right eye, and the “left eye” image is sent to the left eye, creating a 3D effect.

Image: TechCrunch

A great viewing experience means that viewers are able to move freely and enjoy the best image quality at all times. However, movement can be a challenge for autostereoscopic displays, since they need to project a different image to each eye. To overcome this challenge, manufacturers are relying on face tracking. By tracking head position, eye position and even gaze direction, the system can render the video in respect to the viewer’s current view point.

For the viewers, it means that they can move freely in front of the 3D display and enjoy the content regardless of their position.

Tracker requirements

The tracker has to meet specific requirements when it comes to auto 3D displays. Otherwise, the video will not be rendered properly.

First of all, the tracker must be quick and efficient. It must be able to detect and start tracking human faces as soon as they appear in front of the display; for example, when a new viewer appears or when the existing one leaves and then comes back again. If the tracker is too slow, the rendering will be adjusted too late, affecting the quality of the 3D.

Tracking should also be reliable under various real-life conditions. This primarily includes lighting changes and occlusions.

For example, it’s expected that the tracker will work great in the optimal light (such as daylight). However, the room might get darker, or the only light illuminating the viewer’s face could be the one coming from the display itself. Even in such cases, the tracker must work well so that the rendered content can be adjusted according to the viewer’s position.

Furthermore, various occlusions can often occur during the use of the system. This includes glasses, jewelry, tattoos, hair, beards, or any other object that can obstruct the viewer’s face or a part of the face. Such occlusions can be temporary, occurring for a short period of time. For example, the viewer can briefly scratch their face with their hand. They can also be permanent – for example, significant facial hair or tattoos. The tracker should be trained to handle such occlusions as efficiently as possible.

Finally, the hardware used for autostereoscopic displays cannot be too expensive. This means that the tracker should work well with mainstream hardware, such as webcams.

Choosing the right tracker

A reliable tracker can help provide realistic 3D effects without sacrificing user experience. However, in choosing the right tracker for autostereoscopic displays, it’s important to check whether it meets the necessary requirements. The tracker must be quick, efficient, and able to handle various occlusions and real-life conditions. 

FaceTrack – the lightweight face tracking technology developed by Visage Technologies – is a great choice for autostereoscopic displays. It tracks faces, eyes and gaze in real time, helping create an intuitive, glass-free 3D experience without any restrictions when it comes to viewers’ movement of location.

If you’d like to explore FaceTrack yourself, check out our online demo and request your free evaluation license.