Visage Technologies
Face Tracking & Analysis

FaceTrack

FaceTrack package includes face detection and alignment, face tracking and head tracking. Returns 3D head pose, full face data and gaze tracking.

FaceTrack

FaceTrackvisage|SDK integrates a comprehensive range of computer vision technologies related to human faces in an easy-to-use, fully documented Software Development Kit to support a wide area of applications. visage|SDK FaceTrack is capable of tracking the head pose, facial features and gaze for multiple faces in video coming from a video file, camera or other sources. FaceTrack is a fully-configurable powerful package that includes:

 

Main features

  • Tracks multiple faces and facial features in input video, images or in real time
  • Returns 2D and 3D head pose, the coordinates of facial feature points (e.g. chin tip, nose tip, lip corners, 3D face model fitted to the face), mouth contour, chin pose, eyebrow contours, and eye closure and eye rotation (gaze direction)
  • Fully automatic operation
  • Instant initialization: tracking starts immediately when the face is visible
  • Robustly recovers from any losses due to occlusions, face turning away, tracked person coming and going etc.
  • Automatically re-initializes if a new person appears in front of the camera
  • Lightweight technology enables high performance across the whole range of supported platforms

FaceTrack inputs

  • Tracks from any standard camera type (including webcams and mobile phone cameras) or any video file format thanks to the raw image interface
  • Tracks in color or grayscale image, including near-infrared (internal preprocessing is applied on multichannel images)
  • Configurable face search range (minimal and maximal face size to be detected and tracked, relative to image size)
  • Sustainable tracking on faces as small as 30×30 pixels
  • Wide head-pose range: achievable tracking on faces with up to 90 degrees of yaw, 60 degrees of roll and 30 degrees of pitch
  • Extensive tracking distances: for example, for a webcam with resolution 1920×1080 (Logitech C920), tracker can be configured to detect and track faces up to ~7.25 meters from the camera (via min_face_scale parameter set to 0.025, considerable performance tradeoff)

FaceTrack outputs

The tracker offers easy-to-use API for accessing the tracking data on-the-fly during tracking operation. The available data includes:

  • Tracks in color or grayscale
  • Tracks multiple faces in the input video, images or in realtime
  • Tracks faces as small as 30×30 pixels
  • 2D and 3D head pose (translation and rotation), wide head-pose range: up to 90 degrees of yaw, 30 degrees of roll, and 30 degrees of pitch
  • Facial feature coordinates in global 3D space, relative to the head or in 2D image space. The feature points are specified according to the MPEG-4 FBA standard
  • 3D gaze direction (gaze vector)
  • Screen-space gaze coordinates: the point where the user is looking on the screen; requires a short calibration for each user.
  • Eye closure
  • A set of Action Units (e.g. jaw drop, lips stretch, brow raise…) describing the current facial expression; please note that these Action Units are similar but not exactly the same as FACS AUs
  • Standard MPEG-4 FBA Face Animation Parameters
  • 3D model of the face in current pose and expression, returned as single textured 3D triangle mesh; this enables applications such as correct occlusion of virtual objects by the head in AR applications; cutting out the face from the image; drawing the 3D model from a different perspective than the one in the actual video or inserting the face into another video or 3D scene
  • A detailed Technical Specification brochure is available, including measurements of accuracy, performance, code size, data size and memory usage – contact us to request it
wire

Application development/platform availability

Fully configurable tracker

Face tracking is fully configurable through an extensive set of parameters in easily manageable configuration files. The configuration file fully defines the tracker operation, in effect customizing the tracker for a particular application. Default configuration files include:

  • Facial features tracking configuration
  • Head tracking configuration.

Extensive documentation allows users to create own application-specific configurations. The documentation lists and documents all available configuration options. A partial list of main options is here:

  • Performance optimization via trade-off against precision and/or tracked features.
  • Input, work and display resolution settings.
  • Camera mirror option (flip camera image horizontaly).
  • Video file sync option (skip frames to sync or process all frames).
  • Smoothing filters to reduce noise in tracking results.
  • Full control of the 3D head model internally used by the tracker, including the animation rig (this advanced option can potentially be used to completely replace the 3D model by a custom one).

Furthermore, Visage Technologies consulting and custom development services are available to adapt the technology in terms of precision, performance and any other requirements in order to meet the needs of specific applications.

← back  
Go to top