visageSDK
API

visage|SDK provides two APIs: the native C++ API and the Swift API implemented through a wrapper. Both APIs include the following main functionalities:

  • Facial feature tracking
  • Facial feature detection
  • Face analysis
  • Face recognition

All listed APIs use neural networks to process and analyze facial images.

Additional configuration file NeuralNet.cfg is provided in Samples/data folder. It allows for specifying the desired backend and number of threads that the runner will use for inference. Following backend values are supported:

  • AUTO - defaults to TF_CPU
  • TF_CPU - CPU-based inference
  • TF_GPU - GPU-assisted inference

If NeuralNet.cfg is not found, backend will default to AUTO and neural network runner will use 1 thread.

More information about configuring neural network runners can be found in VisageTracker Configuration Manual, chapter 3. Configuring neural network runners.

For the NeuralNet.cfg configuration file to be successfully loaded, Samples/data folder, including its subfolders and files, should keep the same hierarchy as it is in distributed visage|SDK package. Specifically, API expects that the file will be in the same folder as vfa, vfr and vft folders within users' applications.

The following section provides an overview of key functionalities, gives links to most important classes and relevant sample projects, and lists required libraries to link. The detailed information is found in the documentation of each class, reached through links in this section or links on the side menu, or through class reference.

visage|SDK provides three libraries:

  • libVisageVision.a - face detection, alignment and face tracking (requires TFPlugin.framework or TFPlugin-simulator.framework)
  • libVisageGaze.a - screen space gaze tracking (requires libVisageVision.a)
  • libVisageAnalyser.a - face analysis and recognition (requires libVisageVision.a)
  • OpenCV - used as a third-party dependency

Visage tracker tracks multiple faces and facial features in video sequences and outputs 3D head pose, facial expressions, gaze information, eye closure, facial feature points and full 3D face model. The tracker is fully configurable in terms of performance, quality, as well as other options. Common configurations are delivered. Details about configuring the tracker can be found in the VisageTracker Configuration Manual.

Face tracking is demonstrated in VisageTrackerDemo and VisageTrackerUnityDemo sample project.

Main classes

The class VisageFeaturesDetector detects faces and facial features in input images. The results are, for each detected face, the 3D head pose, the coordinates of facial feature points, e.g. chin tip, nose tip, lip corners etc. and 3D face model fitted to the face. The results are returned in one or more FaceData objects, one for each detected face.

The FaceDetectDemo sample project demonstrates facial features detection.

Main classes

The class VisageFaceAnalyser contains face analysis algorithms capable of estimating age, gender and emotion from facial images. For age estimation it returns estimated age, for gender estimation it returns estimated gender and for emotions estimation it returns the probability of each of estimated facial emotions: anger, disgust, fear, happiness, sadness, surprise and neutral.

The FaceDetectDemo sample demonstrates age, gender and emotion estimation.

Main classes

The class VisageFaceRecognition contains a face recognition algorithm capable of measuring similarity between human faces and recognizing a person's identity from frontal facial image (yaw angle approximately from -20 to 20 degrees) by comparing it to previously stored faces.

The FaceDetectDemo sample demonstrates face recognition.

Main classes

  • VisageFaceRecognition: Measures similarity between human faces and recognizes a person's identity from frontal facial images (yaw angle approximately from -20 to 20 degrees).

Liveness system is used to differentiate between a live person in the live video stream, as opposed to a still image. It is used in combination with face recognition to verify that the person is actually in front of the camera. This is accomplished by prompting a person to perform a specific set of facial actions and verifying that the actions have actually been performed. System allows you to use already implemented actions (eyebrows raise, blink, smile), as well as implement your own actions using VisageLivenessAction interface.

Main classes

Swift API

visage|SDK provides a way for users to integrate our native C++ API into their Swift-based applications. The native API is exposed to the application side via an Objective-C-based wrapper which makes its functions visible to Swift through the use of a bridging header. The Swift API can be integrated into the project as a simple drag and drop framework.

Integration

To integrate the visage|SDK Swift Wrapper into your project, follow these steps:

  1. Drag and drop the iOS_SwiftWrapper.framework file located in the lib/ folder of the SDK package into your Xcode project.
  2. On the 'General' tab of your project's settings, change the 'Embed' setting for iOS_SwiftWrapper.framework to 'Embed & Sign'.
  3. On the 'Build Settings' tab, find the 'Objective-C Bridging Header' field and enter the following: *
    (PATH_TO_FRAMEWORK)/iOS_SwiftWrapper.framework/Headers/VisageSDK_bridging_header_iOS.h
  4. Find the 'Header search paths' field and enter the following: *
    (PATH_TO_FRAMEWORK)/iOS_SwiftWrapper.framework/Headers/
  5. Find the 'Framework search paths' field and enter the following: *
    (PATH_TO_FRAMEWORK)

*In these cases, path-to-framework is the physical location of the iOS_SwiftWrapper.framework on your machine

Getting started

You may now begin using visage|SDK in your Swift project. The Swift category in the navigation menu on the left-hand side contains full documentation of all available classes.