visageSDK

API

visage|SDK consists of static libraries libVisageVision, libVisageGaze and libVisageAnalyzer. Additionally, certain other third party libraries need to be installed in order to run sample applications.

The following section provides an overview of key functionalities, gives links to most important classes and relevant sample projects, and lists required libraries to link.

The detailed information is found in the documentation of each class, reached through links in this section or links on the side menu, or through class reference.

visage|SDK provides three libraries:

- libVisageVision.a - face detection, alignment and face tracking, requires:
  - libOVPlugin.so
  - libinference_engine.so
  - libinference_engine_ir_reader.so
  - libinference_engine_lp_transformations.so
  - libinference_engine_transformations.so
  - libMKLDNNPlugin.so
  - libngraph.so
  - libtbb.so
  - libtbb.so.2
  - libtbbmalloc.so
  - libtbbmalloc.so.2
- libVisageGaze.a - screen space gaze tracking, requires:
  - libVisageVision.a
- libVisageAnalyser.a - face analysis and recognition, requires:
  - libVisageVision.a

All listed APIs use neural networks to process and analyze facial images.

Additional configuration file NeuralNet.cfg is provided in Samples/data folder. It allows for specifying the number of threads that the runner will use for inference. Currently, only one runner per platform is available. If NeuralNet.cfg is not found, neural network runner will use 1 thread.

More information about configuring neural network runners can be found in VisageTracker Configuration Manual, chapter 3. Configuring neural network runners.

For the NeuralNet.cfg configuration file to be successfully loaded, Samples/data folder, including its subfolders and files, should keep the same hierarchy as it is in distributed visage|SDK package. Specifically, API expects that the file will be in the same folder as vfa, vfr and vft folders within users' applications.

# Facial features tracking

Visage tracker tracks multiple faces and facial features in video sequences and outputs 3D head pose, facial expressions, gaze direction, facial feature points and textured 3D face model.

The tracker is fully configurable in terms of performance, quality, tracked features and facial actions, as well as other options, allowing in effect a variety of customized trackers suited to different applications. Several common configurations are delivered. Details about configuring the tracker can be found in the VisageTracker Configuration Manual.

Face tracking is demonstrated in VisageTrackerDemo sample project.

Main classes

- VisageTracker: Tracks multiple faces and facial features in video coming from file, camera or other source.
- VisageConfiguration: Used to change, apply, load and save configuration parameters used by VisageTracker.
- FaceData: Used to return tracking results from the tracker.

# Facial feature detection

The class VisageFeaturesDetector detects faces and facial features in input images.

The results are, for each detected face, the 3D head pose, the coordinates of facial feature points, e.g. chin tip, nose tip, lip corners etc. and 3D face model fitted to the face.

The results are returned in one or more FaceData objects, one for each detected face.

The FaceDetectDemo sample project demonstrates facial features detection.

Main classes

- VisageFeaturesDetector: Detects facial features in still images.
- FaceData: Used to return the resulting feature points.

# Facial feature analysis

The class VisageFaceAnalyser contains face analysis algorithms capable of estimating gender and emotion from facial images.

For gender estimation it returns estimated gender and for emotions estimation it returns the probability of each of estimated facial emotions: anger, disgust, fear, happiness, sadness, surprise and neutral.

Main classes

- VisageFaceAnalyser: Face analyser capable of estimating gender and emotion from facial images.
- FaceData: Used to pass the necessary facial image data to VisageFaceAnalyser functions.
- AnalysisData: Used to return analysis results.

# Face recognition

The class VisageFaceRecognition contains a face recognition algorithm capable of measuring similarity between human faces and recognizing a person's identity from frontal facial image (yaw angle approximately from -20 to 20 degrees) by comparing it to previously stored faces.

Main classes

- VisageFaceRecognition: Measures similarity between human faces and recognizes a person's identity from frontal facial images (yaw angle approximately from -20 to 20 degrees).

# Liveness

Liveness system is used to differentiate between a live person in the live video stream, as opposed to a still image. It is used in combination with face recognition to verify that the person is actually in front of the camera. This is accomplished by prompting a person to perform a specific set of facial actions and verifying that the actions have actually been performed. System allows you to use already implemented actions (eyebrows raise, blink, smile), as well as implement your own actions using VisageLivenessAction interface.

Main classes

- VisageLivenessAction: Interface used to implement additional actions.
- VisageLivenessBlink: Implementation of VisageLivenessAction which verifies that blink was performed.
- VisageLivenessSmile: Implementation of VisageLivenessAction which verifies that smile was performed.
- VisageLivenessBrowRaise: Implementation of VisageLivenessAction which verifies that eyebrows raise was performed.