Maximizing tracking performance with OpenVINO toolkit

• 05 May, 2020
• by Valentina Zezelj

Face tracking plays an important role in user awareness applications and AI-powered solutions across industries. Therefore, it’s crucial to make the algorithm operate as quickly and efficiently as possible. Intel’s OpenVINO toolkit helps harness its full potential, enabling new and enhanced use cases.

What is OpenVINO toolkit?

OpenVINO toolkit  (Open Visual Inference and Neural Network Optimization) is a comprehensive toolkit for quickly developing applications and solutions that emulate human vision. Based on Convolutional Neural Networks (CNNs), the toolkit extends CV workloads across Intel® hardware, maximizing performance.

The toolkit is designed for AI researchers, application developers, data scientists, and implementers who require:

• High-performance, deep learning deployments
• Streamlined development and an ease-of-use workflow
• Cross-platform flexibility and scalability with their solutions across Intel® architecture

OpenVINO toolkit accelerates the development of high-performance computer vision and deep learning inference into vision applications.

OpenVINO toolkit integration with visage|SDK

FaceTrack, the face tracking package of visage|SDK, is used to track or detect faces and facial landmarks in images and videos. Its core algorithms are face detection and face alignment.

The face alignment algorithm uses the output of the face detection algorithm in the form of 2D face position and size. Machine learning is used to create a model that can produce 2D landmark locations (face shape) from the image crop containing a face. It is crucial to optimize the model’s inference time in order to obtain real-time performance and low power consumption, especially for applications that require low latency.

To ensure the best run-time/performance across different platforms, we decided to support multiple inference engines by creating a wrapper around them, allowing it to have a single, unified API.

OpenVINO toolkit was selected for desktop and laptop platforms due to Intel’s extensive documentation and support. As a result, specific prerequisites had to be met:

• Visage Technologies’ models needed to be converted into the OpenVINO toolkit format.
• OpenVINO toolkit inference engine had to be wrapped under a consistent API to allow convenient inference engine switching without modifying the SDK code.
• To unlock the full optimization capabilities of the OpenVINO toolkit, the models needed to be calibrated and quantized for computing performance improvements.

The integration was thoroughly tested on Intel laptops. The results have shown that using a highly optimized inference engine such as OpenVINO as an enhancement to the existing solution greatly improves the inference speed of the face tracking algorithm when compared to an implementation based on OpenBLAS. More specifically, it provides approximately four times faster performance.

To sum up, the integration of the OpenVINO toolkit with the visage|SDK provides an excellent solution for computationally efficient face tracking and analysis applications. It also makes optimization and deployment easier for edge devices, which is especially important for use cases such as driver drowsiness detection.

Real-life implementations

Face tracking is a core feature of user awareness applications. During the integration of OpenVINO toolkit with visage|SDK, we focused on two main use cases – driver drowsiness detection for improving road safety, and monitoring user attention to extend laptop and notebook battery life.

– Driver drowsiness monitoring

Optalert, one of the leading companies for drowsiness detection, uses this technology in its driver drowsiness and attentiveness detection solutions. The robustness of the face tracker in this specific use case is essential; driver drowsiness must be detected during both day and night, and wide head angles and rotations must also be supported.

To achieve suitable robustness, several different convolutional neural networks (CNNs) had to be introduced into the face tracker, significantly increasing the processing needs. With Intel® hardware and OpenVINO toolkit, all the requirements were met, and the solution is now operational.

– Laptop user awareness

As Intel announced with their Project Athena, modern laptops and notebooks are expected to have a long battery life with normal usage. One of the most power-hungry components of the laptop is its screen. Face tracking and detection can be used to detect whether a user is present in front of the screen and automatically dim the screen to conserve battery life.

Since many laptops will employ multiple screens, face and head pose tracking can be used to determine which screen the user is focused on in order to dim the currently unused screens. For this application to be power-efficient, the face tracking solution needs to be as fast as possible. By employing the OpenVINO toolkit inference engine, the visage|SDK face tracking solution achieves extremely fast run-time performance.

Get your free demo

The integration of Intel OpenVINO toolkit with visage|SDK is the result of a long-term cooperation between the Visage Technologies team and Intel’s engineers. This article presents the summary of the work performed over the period of 18 months. The official whitepaper with a detailed overview of the project and its test results is available on Intel’s website.

This powerful integration enables you to maximize the performance of applications powered by face tracking. It significantly improves the performance of face tracking, age estimation and face recognition with VNN algorithm on Intel architecture.