VisageTrackerUnityPlugin
Classes | Functions
VisageAnalyserUnityPlugin

Provides high-level functions for using visage|SDK face analysis capabilities in Unity game engine. More...

Classes

struct  ManagedAnalysisData
 Managed equivalent of the native AnalysisData structure, used as a container for all face analysis results. More...
 

Functions

int _initAnalyser (string dataPath)
 Initialises VisageFaceAnalyser. More...
 
void _releaseAnalyser ()
 Releases memory allocated by the face analyser in the _initAnalyser() function. More...
 
VFAReturnCode _analyseImage (int faceIndex, int options, ManagedAnalysisData &results)
 Performs the specified face analysis tasks on a given image and outputs results to ManagedAnalysisData object instance. More...
 
VFAReturnCode _analyseStream (int faceIndex, int options, ManagedAnalysisData &results)
 Performs the specified face analysis tasks on a given frame and outputs results to ManagedAnalysisData object instance. More...
 
bool _prepareDataForAnalysis ()
 Prepares data obtained by the track function.
 

Detailed Description

Provides high-level functions for using visage|SDK face analysis capabilities in Unity game engine.

It implements high-level wrapper functions from visage|SDK that enable usage of visage|SDK face analysis capabilities in Unity C# scripts.

VisageFaceAnalyser contains face analysis algorithms for estimating age, gender and emotion from facial images/frames.

The functionality of VisageFaceAnalyser is dependent on the VisageTracker and it is expected that prior to the call of VisageFaceAnalyser's functions frame is processed with _track() function and the data thus obtained is prepared and forwarded to VisageFaceAnalyser.

Before using VisageFaceAnalyser, it must be initialized by calling the _initAnalyser() function. Afterward, in each frame, the _prepareDataForAnalysis() function is called first, followed by calling _analyseStream() function, otherwise analysis will fail which will be indicated by function's return code.

VisageFaceAnalyser's functionality can be used in a separate thread.

Function Documentation

VFAReturnCode _analyseImage ( int  faceIndex,
int  options,
ManagedAnalysisData &  results 
)

Performs the specified face analysis tasks on a given image and outputs results to ManagedAnalysisData object instance.

This function is primarily intended for performing face analysis on a single image, or consecutive unrelated images. As such, it outputs raw, unfiltered estimation data without smoothing or averaging.

Note
Prior to using this function it is necessary to process the facial image using VisageTracker or VisageFeaturesDetector and pass the frame and obtained data to this function.
Parameters
faceIndexValue between 0 and MAX_FACES-1, used to access the data of a particular tracked face.
optionsBitwise combination of the native VFAFlags enumeration which determines the analysis operations to be performed.
resultsManaged equivalent of the native AnalysisData structure, used as a container for all face analysis results.
Returns
VFAReturnCode Return code indicating status of performed face analysis.
VFAReturnCode _analyseStream ( int  faceIndex,
int  options,
ManagedAnalysisData &  results 
)

Performs the specified face analysis tasks on a given frame and outputs results to ManagedAnalysisData object instance.

This function is primarily intended for performing face analysis on a continuous stream of related frames, such as a video or camera feed. Sampling face analysis data from multiple frames can increase estimation accuracy by averaging the result over multiple frames. Internally, the suitability of frames chosen for analysis is continually evaluted based on head pose and overall tracking quality. This guarantees that the analysis buffer is always working with the best available frames, ensuring highest possible estimation accuracy.

Note
Prior to using this function it is necessary to process the facial image using VisageTracker and pass the frame and obtained data to this function. The bitwise set of analysis operations determined by the 'options' parameter is expected to be constant between subsequent calls to analyseStream(). If the next iteration of analyseStream() is called with a different set of bitwise flags than the previous, the internal buffers will be reset and previously collected analysis data will be lost.

An example of use:

1 public float[] Age = new float[MAX_FACES];
2 public int[] Gender = new int[MAX_FACES];
3 public float[,] Emotions = new float[MAX_FACES, NUM_EMOTIONS];
4 private ManagedAnalysisData[] analysisData = new ManagedAnalysisData[Tracker.MAX_FACES];
5 
6 for(int i = 0; i < MAX_FACES; ++i)
7 {
8  int status = VisageTrackerNative._analyseStream(faceIndex, analyserOptions, ref analysisData[faceIndex]);
9 
10  if(status == VFAReturnCode.NoError)
11  {
12  if (analysisData[faceIndex].ageValid)
13  Age[faceIndex] = analysisData[faceIndex].age;
14  if (analysisData[faceIndex].genderValid)
15  Gender[faceIndex] = analysisData[faceIndex].gender;
16  if (analysisData[faceIndex].emotionsValid)
17  {
18  for (int j = 0; j < NUM_EMOTIONS; j++)
19  Emotions[i, j] = analysisData[faceIndex].emotionProbabilities[j];
20  }
21  }
22 }
Parameters
faceIndexValue between 0 and MAX_FACES-1, used to access the data of a particular tracked face.
optionsBitwise combination of the native VFAFlags enumeration which determines the analysis operations to be performed.
resultsManaged equivalent of the native AnalysisData structure, used as a container for all face analysis results.
Returns
VFAReturnCode Return code indicating status of performed face analysis.
int _initAnalyser ( string  dataPath)

Initialises VisageFaceAnalyser.

This function must be called before using VisageFaceAnalyser by passing it a path to the folder containing the VisageFaceAnalyser data files. Within the Visage|SDK package, this folder is Samples/data/vfa. The VisageFaceAnalyser data folder contains the following subfolders corresponding to the various types of analysis that can be performed:

  • ad: age estimation
  • gd: gender estimation
  • ed: emotion estimation

An example of use:

1 private ManagedAnalysisData[] analysisData = new ManagedAnalysisData[Tracker.MAX_FACES];
2 int analyserInited = VisageTrackerNative._initAnalyser(dataPath);
3 
4 for (int faceIndex = 0; faceIndex < MAX_FACES; faceIndex++)
5 {
6  if ((analyserInited & (int)VFAFlags.VFA_AGE) == (int)VFAFlags.VFA_AGE)
7  {
8  VFAReturnCode status = VisageTrackerNative._analyseStream(faceIndex, (int)VFAFlags.VFA_AGE, ref analysisData[faceIndex]);
9  if ((status == NoError) && analysisData[faceIndex].ageValid)
10  age[faceIndex] = analysisData[faceIndex].age;
11  }
12 }

Note that it is not necessary for all subfolders to be present, only the ones needed by the types of analysis that will be performed. For example, if only age estimation will be performed, then only the ad folder is required. The return value of the function indicates which types of analysis have been successfully initialised. Subsequent attempt to use a type of analysis that was not initialized shall fail; for example, if only ad folder was present at initialization, calling either of the analyse functions with VFAFlags.VFA_EMOTION option will result in non valid emotion estimation. The return value is a bitwise combination of flags indicating which types of analysis are successfully initialized.

Please note that in order for license to be registered, _initializeLicense() method has to be called before.

Parameters
dataPathRelative path from the working directory to the directory that contains VisageFaceAnalyser data files (e.g. Application.streamingAssetsPath + /Visage Tracker/vfa").
Returns
The return value of 0 indicates that no data files are found and no analysis will be possible. In such case, the placement of data files should be verified. A non-zero value indicates that one or more types of analysis are successfully initialized. The specific types of analysis that have been initialized are indicated by a bitwise combination of the following flags: VFA_AGE, VFA_GENDER, VFA_EMOTION.
void _releaseAnalyser ( )

Releases memory allocated by the face analyser in the _initAnalyser() function.

This function is called after the analyser ceases to be used or before the reinitialization of the analyser.