visageSDK
Public Member Functions | List of all members
VisageSDK::VisageFeaturesDetector Class Reference

Faces and facial features detector implementation. More...

#include <VisageFeaturesDetector.h>

Public Member Functions

 VisageFeaturesDetector (const char *detectorConfigFile)
 Constructor. More...
 
 ~VisageFeaturesDetector ()
 Destructor.
 
bool Initialize ()
 Initializes the features detector. More...
 
int detectFaces (VsImage *frame, VsRect *faces, int maxFaces=1, float minFaceScale=0.1f, float maxFaceScale=1.0f, bool useRefinementStep=true)
 Performs face detection in a still image. More...
 
int detectFacialFeatures (VsImage *frame, FaceData *output, int maxFaces=1, float minFaceScale=0.1f, float maxFaceScale=1.0f, bool outputOnly2DFeatures=false)
 Performs faces and facial features detection in a still image. More...
 

Detailed Description

Faces and facial features detector implementation.

This class can be used to detect one or more faces and their facial features in an image. The input is an image bitmap or an image file in one of the supported file formats: JPEG, PNG, BMP or PPM.

VisageFeaturesDetector uses VsImage structure as a container for the input image. VsImage is the image storage class similar to IplImage from OpenCV; it has the same structure and members so it can be used like IplImage. Please refer to OpenCV documentation for details of accessing IplImage data members; the basic members are the size of the image (frame->width, frame->height) and the pointer to the actual pixel data of the image (frame->imageData). VisageFeaturesDetector expects input image to be in the RGB, RGBA or grayscale color format with bit depth set to 8 and with origin in the top left pixel of the image.

There are two ways to create input image:

  1. Create IplImage and set image data using OpenCV. Image should be converted to VsImage before passing it to VisageFeaturesDetector. Please note that OpenCV loads image (imread, cvLoadImage) by default in BGR format. In that case, the input image should be converted before passing it to detector.
  2. visage|SDK provides methods similar to the ones from OpenCV which are used to create input image:
    VsImage *inputImage;
    inputImage = vsCreateImageHeader(vsSize(frameWidth, frameHeight), 8, 3);
    vsSetData(image, (void*)frameData, image->widthStep);
    vsReleaseImageHeader(&inputImage); //release memory

To use the detector, it must first be initialized by calling the function Initialize(). The detector requires a set of data and a configuration file. For more information about required data please consult Initialize() function description.

Methods of detection

Function detectFaces() performs a fast face detection on the image. The result is, for each detected face, a VsRect object containing facial bounding box. VsRect is similar to CvRect from OpenCV; it has the same structure and members.

Function detectFacialFeatures() performs a face and facial features detection on the image. The results is, for each detected face, FaceData object containing 3D head pose, coordinates of facial feature points (e.g. pupils, nose tip, lip corners etc.), 3D face model fitted to the face and more. Please refer to the FaceData documentation for detailed description of returned data.

Implemented in libVisageVision.lib

Demonstrated in FaceDetector sample project.

Definition at line 106 of file VisageFeaturesDetector.h.

Constructor & Destructor Documentation

◆ VisageFeaturesDetector()

VisageSDK::VisageFeaturesDetector::VisageFeaturesDetector ( const char *  detectorConfigFile)

Constructor.

Creates detector object with specified configuration file.

Configuration file is used to modify detector behavior and set paths to the required data files. Default detector configuration file Face Detector.cfg is provided in the Samples/data folder.

To successfully initialize VisageFeaturesDetector, provide path to the configuration file that is located in folder with the following folder structure and data files:

  • vft folder that contains algorithm and features specific files. NOTE: The subfolder structure and names must be retained.
    • Required folders and algorithm data files:
      • ff/ *
      • fa/ *
        • NOTE: If landmarks refinement (configuration parameter refine_landmarks [0]) is disabled, d2.tflite is not needed.
      • fm/ containing .fdp and .wfm file corresponding to the 3D model (configuration parameters *_fitting_model, *_fitting_fdp)

Please note that data files packaged within Android apk file cannot be read by our API so it is necessary to copy all required files to some other destination, e.g. application's data folder. Example of how this is done can be found in all our samples. For example, you can take look at FaceDetectDemo sample where copying is done in StartupActivity.java, in method copyAssets().

Parameters
detectorConfigFilethe name of the detector configuration file (.cfg), relative to the current working directory, e.g. "Samples/data/Face Detector.cfg", considering visageSDK as a working directory.

Member Function Documentation

◆ detectFaces()

int VisageSDK::VisageFeaturesDetector::detectFaces ( VsImage *  frame,
VsRect *  faces,
int  maxFaces = 1,
float  minFaceScale = 0.1f,
float  maxFaceScale = 1.0f,
bool  useRefinementStep = true 
)

Performs face detection in a still image.

The algorithm detects one or more faces. For each detected face a square facial bounding box is returned.

The results are returned in form of VsRect objects. An array of VsRect objects passed to this method as output parameter should be allocated to maxFaces size. For example:

VsRect* faces = new VsRect[maxFaces];
n = this->m_Detector->detectFaces(image, faces, maxFaces);

After this call, n contains the number of detected faces. The first n members of the faces array are filled with resulting bounding boxes for each detected face. If maxFaces is smaller than the number of faces actually detected in the image, the function will return only first maxFaces detected faces.

VsImage is the image storage class similar to IplImage from OpenCV, it has the same structure and members so it can be used like IplImage. Please refer to OpenCV documentation for details of accessing IplImage data members; the basic members are the size of the image (frame->width, frame->height) and the pointer to the actual pixel data of the image (frame->imageData).

VisageFeaturesDetector expects input image to be in the RGB, RGBA or grayscale color space with bit depth set to 8 and with origin in the top left pixel of the image.

Note that the input image is internally converted to grayscale.

Parameters
framethe input image.
facespointer to an array of VsRect objects in which the results will be returned.
maxFacesmaximum number of faces to be detected
minFaceScalescale of smallest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
maxFaceScalescale of largest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
useRefinementStepif set to true, additional refinement algorithm will be used resulting with more precise facial bounding boxes and lower FPR, but higher detection time
Returns
number of detected faces (0 or more)

◆ detectFacialFeatures()

int VisageSDK::VisageFeaturesDetector::detectFacialFeatures ( VsImage *  frame,
FaceData *  output,
int  maxFaces = 1,
float  minFaceScale = 0.1f,
float  maxFaceScale = 1.0f,
bool  outputOnly2DFeatures = false 
)

Performs faces and facial features detection in a still image.

The algorithm detects one or more faces and their features. The results are, for each detected face, the 3D head pose, gaze direction, eye closure, the coordinates of facial feature points (e.g. chin tip, nose tip, lip corners etc.) and 3D face model fitted to the face. If outputOnly2DFeatures is set, only the 2D feature points will be returned.

The results are returned in form of FaceData objects. An array of FaceData objects passed to this method as output parameter should be allocated to maxFaces size. For example:

FaceData* data = new FaceData[maxFaces];
n = this->m_Detector->detectFacialFeatures(image, data, maxFaces, minFaceScale);

After this call, n contains the number of faces actually detected. The first n members of the data array are filled with resulting data for each detected face. Please refer to the FaceData documentation for detailed description of returned parameters. If maxFaces is smaller than the number of faces actually present in the image, the function will return only first maxFaces detected faces.

VsImage is the image storage class similar to IplImage from OpenCV, it has the same structure and members so it can be used like IplImage. Please refer to OpenCV documentation for details of accessing IplImage data members; the basic members are the size of the image (frame->width, frame->height) and the pointer to the actual pixel data of the image (frame->imageData).

VisageFeaturesDetector expects input image to be in the RGB, RGBA or grayscale color space with bit depth set to 8 and with origin in the top left pixel of the image.

Note that the input image is internally converted to grayscale.

Parameters
framethe input image.
outputpointer to an array of FaceData objects in which the results will be returned.
maxFacesmaximum number of faces to be detected
minFaceScalescale of smallest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
maxFaceScalescale of largest face to be searched for, defined as decimal fraction [0.0 - 1.0] of input image size (min(width, height))
outputOnly2DFeaturesif set, detection time will be reduced and only FeaturePoints2D will be returned
Returns
number of detected faces (0 or more)
See also
FaceData

◆ Initialize()

bool VisageSDK::VisageFeaturesDetector::Initialize ( )

Initializes the features detector.

Returns
true if successful

The documentation for this class was generated from the following file: