visageSDK
Public Member Functions | List of all members
VisageSDK::VisageFaceAnalyser Class Reference

VisageFaceAnalyser contains face analysis algorithms capable of estimating age, gender and emotion from facial images. More...

#include <VisageFaceAnalyser.h>

Public Member Functions

 VisageFaceAnalyser ()
 Constructor.
 
 ~VisageFaceAnalyser ()
 Destructor.
 
int init (const char *dataPath)
 Initialise VisageFaceAnalyser. More...
 
VFAReturnCode analyseImage (const VsImage *image, const FaceData &faceData, const int options, AnalysisData &results)
 Performs face analysis tasks specified by options parameter on a given image and outputs results to an AnalysisData object instance. More...
 
VFAReturnCode analyseStream (const VsImage *frame, const FaceData &faceData, const int options, AnalysisData &results, const int faceIndex=0)
 Performs face analysis tasks specified by options parameter on a given frame and outputs results to an AnalysisData object instance. More...
 
void resetStreamAnalysis (const int faceIndex=-1)
 Resets collected face analysis data. More...
 

Detailed Description

VisageFaceAnalyser contains face analysis algorithms capable of estimating age, gender and emotion from facial images.

Before using VisageFaceAnalyser, it must be initialized using the function init(). After the initialization the following types of analysis can be used:

Face analysis may be performed on an individual image, or on a sequential stream of images (such as a video or camera feed). The two corresponding functions are:

analyseImage()Perform face analysis on a single image
analyseStream()Perform face analysis on a sequential stream (utilizes score-based selection of ideal candidate frames to deliver a high-precision averaged result)

VisageFaceAnalyser uses VsImage structure as a container for the input image. Please refer to the VsImage structure documentation for details of accessing its data members; the basic members are the size of the image (frame->width, frame->height) and the pointer to the actual pixel data of the image (frame->imageData). All of the estimation methods expect grayscale, RGB or RGBA image. Please note that for the most efficient usage, the grayscale image format should be used. Grayscale format is a default image format for the estimation methods and conversion from other valid image formats is done internally.

The following is an example of how to create an input image using methods provided by visage|SDK:

VsImage *inputImage;
const int nChannels = 3;
inputImage = vsCreateImageHeader(vsSize(frameWidth, frameHeight), VS_DEPTH_8U, nChannels);
int frameWidthStep = frameWidth * nChannels; //If no padding, otherwise the widthStep should include the padding
vsSetImageData(image, frameData, frameWidthStep);
vsReleaseImageHeader(&inputImage); //release memory


Definition at line 163 of file VisageFaceAnalyser.h.

Member Function Documentation

VFAReturnCode VisageSDK::VisageFaceAnalyser::analyseImage ( const VsImage *  image,
const FaceData &  faceData,
const int  options,
AnalysisData &  results 
)

Performs face analysis tasks specified by options parameter on a given image and outputs results to an AnalysisData object instance.

This function is primarily intended for performing face analysis on a single image, or consecutive unrelated images. As such, it outputs raw, unfiltered estimation data without smoothing or averaging.

Note
Prior to using this function it is necessary to process the facial image using VisageTracker or VisageFeaturesDetector and pass the frame and obtained data to this function. When using data from VisageTracker, only data obtained when tracking status was TRACK_STAT_OK should be passed.

An example of usage with OpenCV, estimate gender and age on all the faces detected in an image file:

VisageFeaturesDetector* detector = new VisageFeaturesDetector("Face Detector.cfg");
bool init = detector->Initialize();
const char* dataPath = "vfa";
int isInitialized = analyser->init(dataPath);
IplImage* cvImg = cvLoadImage(filePath, CV_LOAD_IMAGE_GRAYSCALE);
VsImage* inputImage = vsCreateImageHeader(vsSize(cvImg->width, cvImg->height), 8, 1);
vsSetData(inputImage, cvImg->imageData, cvImg->widthStep);
const int MAX_FACES = 4;
const int options = VFA_AGE | VFA_GENDER | VFA_EMOTION;
if ((isInitialized & options) == options)
{
FaceData* data = new FaceData[MAX_FACES];
int nFaces = detector->detectFacialFeatures(inputImage, data, MAX_FACES);
AnalysisData results[MAX_FACES];
VFAReturnCode status;
for (int i = 0; i < nFaces; i++)
{
status = analyser->analyseImage(reinterpret_cast<VsImage*>(cvImg), data[i], options, results[i]);
if (status == VFAReturnCode::NoError)
{
if (results[i].ageValid)
// use age
if (results[i].genderValid)
// use gender
if (results[i].emotionsValid)
// use emotion
}
}
delete[] data;
}
vsReleaseImageHeader(&inputImage); //release memory
cvReleaseImage(&cvImg); //release memory
delete detector;
Parameters
imageVsImage container that contains the input grayscale, RGB or RGBA image.
faceDataFaceData obtained by a previous successful tracking (TRACK_STAT_OK) or detection operation with VisageTracker or VisageFeaturesDetector.
optionsBitwise combination of VFAFlags which determines the analysis operations to be performed.
resultsAnalysisData struct containing success flags for individual operations and their assorted results.
Returns
VFAReturnCode value indicating the status of the performed analysis.
VFAReturnCode VisageSDK::VisageFaceAnalyser::analyseStream ( const VsImage *  frame,
const FaceData &  faceData,
const int  options,
AnalysisData &  results,
const int  faceIndex = 0 
)

Performs face analysis tasks specified by options parameter on a given frame and outputs results to an AnalysisData object instance.

This function is primarily intended for performing face analysis on a continuous stream of related frames containing the same person, such as a video or camera feed. Sampling face analysis data from multiple frames can increase estimation accuracy by averaging the result over multiple frames. Internally, the suitability of frames chosen for analysis is continually evaluted based on head pose and overall tracking quality. This guarantees that the analysis buffer is always working with the best available frames, ensuring highest possible estimation accuracy.

Important notes:
This method should only be called with FaceData obtained from a successful tracking operation that returned TRACK_STAT_OK tracking status.

If options parameter is changed between subsequent calls to analyseStream(), the internal state will be reset and previously collected analysis data will be lost. For optimal results options parameter should remain constant during single stream analysis session.

If a new person in the continuous stream replaces the old one, it is neccessary to call resetStreamAnalysis() method otherwise results will not be correct as analyseStream() method has no capability of differentiating faces.

Note
Prior to using this function it is necessary to process the facial image using VisageTracker and pass the frame and obtained data to this function.

An example of usage with OpenCV, estimate gender and age on all the faces tracked in a video file:

VisageTracker* tracker = new VisageTracker("Facial Features Tracker.cfg");
const char *dataPath = "vfa";
int isInitialized = analyser->init(dataPath);
const int MAX_FACES = 1;
FaceData data;
const int options = VFA_AGE | VFA_GENDER | VFA_EMOTION;
AnalysisData analysisData;
cap = cvCaptureFromFile(filePath);
IplImage* videoFrame;
if(isInitialized & options)
{
for (int i = 0; i < 30; i++)
{
videoFrame = cvQueryFrame(cap);
int width = cvGetCaptureProperty(cap, CV_CAP_PROP_FRAME_WIDTH);
int height = cvGetCaptureProperty(cap, CV_CAP_PROP_FRAME_HEIGHT);
int timeStamp = cvGetCaptureProperty(cap, CV_CAP_PROP_POS_MSEC);
int* trackerStatus = tracker->track(width, height, videoFrame->imageData, &data, VISAGE_FRAMEGRABBER_FMT_BGR, VISAGE_FRAMEGRABBER_ORIGIN_TL, videoFrame->widthStep, timeStamp, MAX_FACES);
if (trackerStatus[0] == TRACK_STAT_OK)
{
VsImage* frameRGB = vsCreateImage(vsSize(width, height), VS_DEPTH_8U, 3);
vsCvtColor(videoFrame, frameRGB, VS_BGR2RGB);
VFAReturnCode status = analyser->analyseStream(reinterpret_cast<VsImage*>(frameRGB), data, options, analysisData);
if (status == VFAReturnCode::NoError)
{
if (results[i].ageValid)
// use age
if (results[i].genderValid)
// use gender
if (results[i].emotionsValid)
// use emotion
}
vsReleaseImage(&frameRGB);
}
}
}
delete tracker;
delete analyser;
Parameters
frameVsImage container that contains the input grayscale, RGB or RGBA frame.
faceDataFaceData obtained by a previous successful tracking operation with VisageTracker (TRACK_STAT_OK).
optionsBitwise combination of VFAFlags which determines the analysis operations to be performed.
resultsAnalysisData struct containing success flags for individual operations and their assorted results.
faceIndexIndex of the face for which analysis should be performed.
Returns
VFAReturnCode value indicating the status of the performed analysis.
int VisageSDK::VisageFaceAnalyser::init ( const char *  dataPath)

Initialise VisageFaceAnalyser.

This function must be called before using VisageFaceAnalyser by passing it a path to the folder containing the VisageFaceAnalyser data files. Within the visage|SDK package, this folder is Samples/data/vfa*. When implementing applications, developers may copy this folder as part of their application. The path is given relative to the current working directory at the time of calling the init() function.

The VisageFaceAnalyser data folder contains the following subfolders corresponding to the various types of analysis that can be performed:

  • ad: age estimation
  • gd: gender estimation
  • ed: emotion estimation

Note that it is not neccessary for all subfolders to be present, only the ones needed by the types of analysis that will be performed. For example, if only gender estimation will be performed, then only the gd folder is required.

The return value of the init() function indicates which types of analysis have been successfully initialised. Subsequent attempt to use a type of analysis that was not initialized shall fail; for example, if only gd folder was present at initialization, attempt to estimate emotions shall fail.

The return value is a bitwise combination of flags indicating which types of analysis are successfully initialized.

Parameters
dataPathrelative path from the working directory to the directory that contains VisageFaceAnalyser data files.
Returns
The return value of 0 indicates that no data files are found and no analysis will be possible. In such case, the placement of data files should be verified. A non-zero value indicates that one or more types of analysis are successfully initialized. The specific types of analysis that have been initialized are indicated by a bitwise combination of the flags from VFAFlags.
void VisageSDK::VisageFaceAnalyser::resetStreamAnalysis ( const int  faceIndex = -1)

Resets collected face analysis data.

Erases age, gender and emotion data collected up to this point.

This is intended to be used in cases when a new person replaces the old one in the same continous input stream.

If an index parameter is specified, only data for that specific face is erased. If no parameter is specified, data for all faces is erased.

Parameters
faceIndexIndex of the face for which analysis data should be reset

The documentation for this class was generated from the following file: