|
visageSDK
|
Provides high-level functions for using visage|SDK face tracking capabilities in Unity game engine. More...
Functions | |
| void | _initializeLicense (string license) |
| Initializes the license. More... | |
| void | _grabFrame () |
| Grabs current frame from the camera. More... | |
| void | _openCamera (int orientation, int deviceID, int width, int height, int isMirrored) |
| Initializes camera using native OSXInput library with given orientation, camera width and height, deviceID and isMirrored parameter for possible frame mirroring. More... | |
| void | _closeCamera () |
| Closes camera and clears memory allocated by the camera API. | |
| void | _bindTexture (int texID) |
| Binds a texture with the given native hardware texture id through OpenGL. More... | |
| void | _bindTextureMetal (void *texID) |
| Binds a texture with the given native hardware texture id through OpenGL. More... | |
| void | _initTracker (string config, int numFaces) |
| Initialises the tracker with configuration file and a desired number of tracked faces. More... | |
| void | _releaseTracker () |
| Releases memory allocated by the tracker in the _initTracker() function. More... | |
| void | _track () |
| Performs face tracking on current frame. More... | |
| void | _getTrackerStatus (int[] tStatus) |
| Returns tracker status. More... | |
| float | _getTrackingQuality (int faceIndex) |
| Estimated tracking quality for the current frame for the given faceIndex. More... | |
| float | _getFrameRate () |
| Returns the frame rate of the tracker, in frames per second, measured over last 10 frames. | |
| int | _getTimeStamp () |
| Returns timestamp of the current video frame, in milliseconds, measured from the moment when tracking started. | |
| int | _getFaceScale (int faceIndex) |
| Returns size of face bounding box in pixels for the given faceIndex. More... | |
| void | _getFaceBoundingBox (VsRect *rectangle, int faceIndex) |
| Returns the face bounding box for the given faceIndex. More... | |
| void | _getHeadTranslation (float *translation, int faceIndex) |
| Returns the current head translation in meters for the given faceIndex. More... | |
| void | _getHeadRotation (float *rotation, int faceIndex) |
| Returns the current head rotation in radians for the given faceIndex. More... | |
| void | _getHeadRotationApparent (float *rotationApparent, int faceIndex) |
| Returns the current apparent head rotation in radians for the given faceIndex. More... | |
| int | _getFaceModelVertexCount () |
| Returns number of vertices in the 3D face model. More... | |
| int | _getFaceModelTriangleCount () |
| Returns number of triangles in the 3D face model. More... | |
| void | _getFaceModelTriangles (int *triangles, int faceIndex) |
| Returns triangles coordinates of the 3D face model. More... | |
| void | _getFaceModelVertices (float *vertices, int faceIndex) |
| Returns vertex coordinates of the 3D face model. More... | |
| void | _getFaceModelVerticesProjected (float *verticesProjected, int faceIndex) |
| Returns projected (image space) vertex coordinates. More... | |
| void | _getFaceModelTextureCoords (float *texCoord, int faceIndex) |
| Returns texture coordinates. More... | |
| int | _getFP_START_GROUP_INDEX () |
| Index of the first feature point group. More... | |
| int | _getFP_END_GROUP_INDEX () |
| Index of the last feature point group. More... | |
| void | _getGroupSizes (int *groupSizeArray, int length) |
| Get the size of all feature point groups. More... | |
| void | _getCameraInfo (out float focus, out int width, out int height) |
| Returns camera info. More... | |
| int | _getActionUnitCount () |
| Returns the action unit count. More... | |
| string [] | _getActionUnitName (int auIndex) |
| Returns the name of the action unit with the specified index. More... | |
| bool | _getActionUnitUsed (int auIndex) |
| Returns true if the action unit is used. More... | |
| void | _getActionUnitValues (float[] values, int faceIndex) |
| Returns all action unit values. More... | |
| bool | _getGazeDirection (float[] direction, int faceIndex) |
| Returns the gaze direction for the given faceIndex. More... | |
| bool | _getGazeDirectionGlobal (float[] direction, int faceIndex) |
| Returns the global gaze direction for the given faceIndex. More... | |
| bool | _getEyeClosure (float[] closure, int faceIndex) |
| Returns eye closure value for the given faceIndex. More... | |
| bool | _getIrisRadius (float[] radius, int faceIndex) |
| Returns iris radius values in px for the given faceIndex. More... | |
| void | _getAllFeaturePoints2D (float *featurePointArray, int length, int faceIndex) |
| Returns for all feature points the position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _getAllFeaturePoints3D (float *featurePointArray, int length, int faceIndex) |
| Returns for all feature points the global 3D position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _getAllFeaturePoints3DRelative (float *featurePointArray, int length, int faceIndex) |
| Returns for all feature points the position in 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _getFeaturePoints2D (int N, int *groups, int *indices, float *positions, int *defined, int *detected, float *quality, int faceIndex) |
| Returns the feature points position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _getFeaturePoints3D (int N, int *groups, int *indices, float *positions, int *defined, int *detected, float *quality, int faceIndex) |
| Returns the global 3D feature point position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _getFeaturePoints3DRelative (int N, int *groups, int *indices, float *positions, int *defined, int *detected, float *quality, int faceIndex) |
| Returns the 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex. More... | |
| void | _setTrackerConfiguration (string trackerConfigFile, bool au_fitting_disabled, bool mesh_fitting_disabled) |
| Sets tracking configuration. More... | |
| void | _setIPD (float IPDvalue) |
| Sets the inter pupillary distance. More... | |
| float | _getIPD () |
| Returns the current inter pupillary distance (IPD) setting. More... | |
Provides high-level functions for using visage|SDK face tracking capabilities in Unity game engine.
Additionally, provides high-level functions for camera management such as _openCamera(), _closeCamera() and _getCameraInfo().
It implements high-level wrapper functions around key functionalities from visage|SDK that enable usage of visage|SDK face tracking capabilities in Unity C# scripts.
VisageTracker is a face tracker capable of tracking the head pose and facial features for multiple faces in video coming from a video file, camera or other sources. Maximum number of faces that can be tracked is limited to 20 within native plugin library with MAX_FACES parameter. Parametar faceIndex with value between 0 and MAX_FACES is passed to the functions within native plugin library to obtain the data for a particular tracked face. All functions that return the data obtained from the tracker, internally check the tracker status as well as the value of the parameter faceIndex.
The tracker is fully configurable through comprehensive tracker configuration files.visage|SDK contains optimal configurations for common uses such as head tracking and facial features tracking.Please refer to the VisageTracker Configuration Manual for the list of available configurations. The VisageTracker Configuration Manual provides full detail on all available configuration options, allowing to customize the tracker in terms of performance, quality, and other options.
High-level functions offer the following outputs:
The tracker can apply a smoothing filter to tracking results to reduce the inevitable tracking noise. Smoothing factors are adjusted separately for different parts of the face.The smoothing settings in the supplied tracker configurations are adjusted conservatively to avoid delay in tracking response, yet provide reasonable smoothing. For further details please see the smoothing_factors parameter array in the VisageTracker Configuration Manual.
Frames are fetched natively using OSXInput library by calling _grabFrame() function and are applied to the Unity-created texture using OpenGL in _bindTexture() function. Function _grabFrame() needs to be called before each sequential call to the _track() function to obtain a new frame.
Afterwords, high-level function such as _getFeaturePoints2D() are called to obtain tracker results for the given frame.
The tracker requires a set of data and configuration files, available in Samples/data folder. For most Unity applications usage of Application.streamingAssetsPath folder is encouraged.
| void _bindTexture | ( | int | texID | ) |
Binds a texture with the given native hardware texture id through OpenGL.
| texID | The native hardware ID of the texture to bind to. |
| void _bindTextureMetal | ( | void * | texID | ) |
Binds a texture with the given native hardware texture id through OpenGL.
| texID | The native hardware ID of the texture to bind to. |
| int _getActionUnitCount | ( | ) |
Returns the action unit count.
| string [] _getActionUnitName | ( | int | auIndex | ) |
Returns the name of the action unit with the specified index.
| auIndex | Index of the action unit for the given faceIndex. |
| bool _getActionUnitUsed | ( | int | auIndex | ) |
Returns true if the action unit is used.
| auIndex | Index of the action unit. |
| void _getActionUnitValues | ( | float [] | values, |
| int | faceIndex | ||
| ) |
Returns all action unit values.
An example of use:
| values | Array that will be filled with action unit values. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getAllFeaturePoints2D | ( | float * | featurePointArray, |
| int | length, | ||
| int | faceIndex | ||
| ) |
Returns for all feature points the position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
For example, first feature point is 2.1, therefore first five values in featurePointArray array will be filled with the following values:
An example of use in Unity:
| featurePointArray | array in which the position in normalized 2D screen coordinates, indication whether the point is defined and detected and feature point quality will be stored. |
| length | length of featurePointArray. |
| faceIndex | value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getAllFeaturePoints3D | ( | float * | featurePointArray, |
| int | length, | ||
| int | faceIndex | ||
| ) |
Returns for all feature points the global 3D position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:
An example of use in Unity:
NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
| featurePointArray | array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored. |
| length | length of featurePointArray. |
| faceIndex | value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getAllFeaturePoints3DRelative | ( | float * | featurePointArray, |
| int | length, | ||
| int | faceIndex | ||
| ) |
Returns for all feature points the position in 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
For example, first feature point is 2.1, therefore first six values in featurePointArray array will be filled with the following values:
An example of use in Unity:
NOTE: quality information is returned exclusively within _getAllFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
| featurePointArray | array in which the position in x, y and z coordinates, indication whether the point is defined and detected and feature point quality will be stored. |
| length | length of featurePointArray. |
| faceIndex | value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getCameraInfo | ( | out float | focus, |
| out int | width, | ||
| out int | height | ||
| ) |
Returns camera info.
This function should be called after _openCamera() and _track() functions are preformed, otherwise it will join -1 value to parametar focus and 0 to parameters width and height.
| focus | Parameter to which focal distance of the camera will be assigned. |
| width | Parameter to which width of the frame will be assigned. |
| height | Parameter to which height of the frame will be assigned. |
| bool _getEyeClosure | ( | float [] | closure, |
| int | faceIndex | ||
| ) |
Returns eye closure value for the given faceIndex.
Discrete eye closure value is expressed with two values: index 0 represents closure of left eye. Index 1 represents closure of right eye. Value of 1 represents open eye. Value of 0 represents closed eye.
An example of use:
| closure | Array that will be filled with eye closure values: index 0 represents closure of left eye. Index 1 represents closure of right eye. Value of 1 represents open eye. Value of 0 represents closed eye. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFaceBoundingBox | ( | VsRect * | rectangle, |
| int | faceIndex | ||
| ) |
Returns the face bounding box for the given faceIndex.
Parameter rectangle is a VsRect structure whose members represent x and y coordinates of the upper-left corner and the width and height of the box. All members are expressed in pixels. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill rectangle members with -1 values.
| rectangle | Structure where face bounding box's coordinates and size will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFaceModelTextureCoords | ( | float * | texCoord, |
| int | faceIndex | ||
| ) |
Returns texture coordinates.
Texture coordinates for the 3D face model. A pair of u, v coordinates for each vertex.
Function returns texture coordinates in the vertexNumber*2-dimensional texCoord array, where vertexNumber is value obtained from _getFaceModelVertices() function, for the particular faceIndex.
An example of use:
| texCoord | Array in which values of texture coordinates will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| int _getFaceModelTriangleCount | ( | ) |
Returns number of triangles in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.
| void _getFaceModelTriangles | ( | int * | triangles, |
| int | faceIndex | ||
| ) |
Returns triangles coordinates of the 3D face model.
Each triangle is described by three indices into the list of vertices.
Function returns triangles coordinates in the triangleNumber*3-dimensional triangles array, where triangleNumber is value obtained from _getFaceModelTriangleCount() function, for the particular faceIndex.
| triangles | Array in which values of triangle coordinates will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| int _getFaceModelVertexCount | ( | ) |
Returns number of vertices in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will be set to 0.
| void _getFaceModelVertices | ( | float * | vertices, |
| int | faceIndex | ||
| ) |
Returns vertex coordinates of the 3D face model.
The coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-axis points up and z-axis points into the face.
Function returns vertices position in the vertexNumber*3-dimensional vertices array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use:
| vertices | Array in which the values of vertex coordinates will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFaceModelVerticesProjected | ( | float * | verticesProjected, |
| int | faceIndex | ||
| ) |
Returns projected (image space) vertex coordinates.
The 2D coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns vertices position in the vertexNumber*2-dimensional verticesProjected array, where vertexNumber is value obtained from _getFaceModelVertexCount() function, for the particular faceIndex.
An example of use:
| verticesProjected | Array in which the values of projected vertex coordinates will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| int _getFaceScale | ( | int | faceIndex | ) |
Returns size of face bounding box in pixels for the given faceIndex.
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFeaturePoints2D | ( | int | N, |
| int * | groups, | ||
| int * | indices, | ||
| float * | positions, | ||
| int * | defined, | ||
| int * | detected, | ||
| float * | quality, | ||
| int | faceIndex | ||
| ) |
Returns the feature points position in normalized 2D screen coordinates, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Function returns position for N feature points in the N*2-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:
| N | number of feature points. |
| groups | N-dimensional array of the groups of the feature points. |
| indices | N-dimensional array of the indices of the feature points within the groups. |
| positions | N*2-dimensional array filled with resulting facial feature points positions. |
| defined | N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. |
| detected | N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. |
| quality | N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. |
| faceIndex | value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFeaturePoints3D | ( | int | N, |
| int * | groups, | ||
| int * | indices, | ||
| float * | positions, | ||
| int * | defined, | ||
| int * | detected, | ||
| float * | quality, | ||
| int | faceIndex | ||
| ) |
Returns the global 3D feature point position in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:
NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
| N | number of feature points. |
| groups | N-dimensional array of the groups of the feature points. |
| indices | N-dimensional array of the indices of the feature points within the groups. |
| positions | N*3-dimensional array filled with resulting facial feature points positions. |
| defined | N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. |
| detected | N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. |
| quality | N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. |
| faceIndex | value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getFeaturePoints3DRelative | ( | int | N, |
| int * | groups, | ||
| int * | indices, | ||
| float * | positions, | ||
| int * | defined, | ||
| int * | detected, | ||
| float * | quality, | ||
| int | faceIndex | ||
| ) |
Returns the 3D coordinates relative to the face origin, placed at the center between eyes in meters, indicator whether the point is defined, indicator whether the point is detected or estimated and feature point quality for the given faceIndex.
Returned coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face.
Function returns position for N feature points in the N*3-dimensional positions array, an indication of whether the point is defined as 0 and 1 in the N-dimensional defined array and detected as 0 and 1 in the N-dimensional detected array. The quality of each point as value between 0 and 1 is returned in quality array. For each point, its group and index are specifically listed in the groups and indices arrays.
An example of use:
NOTE: quality information is returned exclusively within _getFeaturePoints2D() function. Therefore, this function will always return -1 for the quality of each point.
| N | Number of feature points. |
| groups | N-dimensional array of the groups of the feature points. |
| indices | N-dimensional array of the indices of the feature points within the groups. |
| positions | N*3-dimensional array filled with resulting facial feature points positions. |
| defined | N-dimensional array filled for every feature point with a value that indicates whether the point is defined. Value 1 is assigned to defined points, and 0 to undefined points. |
| detected | N-dimensional array filled for every feature point with a value that indicates whether the point is detected. Value 1 is assigned to detected points, and 0 to undetected points. |
| quality | N-dimensional array filled for every feature point with a value from 0 to 1, where 0 is the worst and 1 is the best quality. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| int _getFP_END_GROUP_INDEX | ( | ) |
Index of the last feature point group.
| int _getFP_START_GROUP_INDEX | ( | ) |
Index of the first feature point group.
| bool _getGazeDirection | ( | float [] | direction, |
| int | faceIndex | ||
| ) |
Returns the gaze direction for the given faceIndex.
Direction is expressed with two values x and y, in radians. Values (0, 0) correspond to person looking straight. X is the horizontal rotation with positive values corresponding to person looking to his/her left. Y is the vertical rotation with positive values corresponding to person looking down.
An example of use:
Note that gaze direction is influenced with process_eyes parameter from tracker configuration file.
| direction | Array that will be filled with direction expressed with two values x and y. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| bool _getGazeDirectionGlobal | ( | float [] | direction, |
| int | faceIndex | ||
| ) |
Returns the global gaze direction for the given faceIndex.
Global gaze direction is the current estimated gaze direction relative to the camera axis. Direction is expressed with three values determining the rotations around the three axes x, y and z, i.e. pitch, yaw and roll.
An example of use:
Note that global gaze direction influenced with process_eyes parameter from tracker configuration file.
| direction | Array that will be filled with direction expressed with three values determining the rotations around the three axes x, y and z. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getGroupSizes | ( | int * | groupSizeArray, |
| int | length | ||
| ) |
Get the size of all feature point groups.
| groupSizeArray | Array in which the size of each group will be stored. |
| length | Length of groupSizeArray array. |
| void _getHeadRotation | ( | float * | rotation, |
| int | faceIndex | ||
| ) |
Returns the current head rotation in radians for the given faceIndex.
Parameter rotation is 3-dimensional array which will be filled x, y and z head rotation coordinates for the particular face given with faceIndex parameter.
This is the estimated rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero rotation (values 0, 0, 0) corresponds to the face looking straight ahead along the camera axis. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image. The values are in radians.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill rotation array with 0 values.
| rotation | 3D array in which the head rotation values will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getHeadRotationApparent | ( | float * | rotationApparent, |
| int | faceIndex | ||
| ) |
Returns the current apparent head rotation in radians for the given faceIndex.
Parameter rotation is 3-dimensional array which will be filled x, y and z head rotation coordinates for the particular face given with faceIndex parameter.
This is the estimated apparent rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero rotation (values 0, 0, 0) corresponds to the face looking straight into the camera i.e. a frontal face. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image. The values are in radians.
This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill rotation array with 0 values.
| rotationApparent | 3D array in which the apparent head rotation values will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getHeadTranslation | ( | float * | translation, |
| int | faceIndex | ||
| ) |
Returns the current head translation in meters for the given faceIndex.
Parameter translation is 3-dimensional array which will be filled x, y and z head translation coordinates for the particular face given with faceIndex parameter. The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera. The reference point on the head is in the center between the eyes. This variable is set only while tracker is tracking (TRACK_STAT_OK), otherwise it will fill translation array with -10000 values.
| translation | 3D array in which the head translation values will be stored. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| float _getIPD | ( | ) |
Returns the current inter pupillary distance (IPD) setting.
IPD setting is used by the tracker to estimate the distance of the face from the camera.
| bool _getIrisRadius | ( | float [] | radius, |
| int | faceIndex | ||
| ) |
Returns iris radius values in px for the given faceIndex.
This variable is set only while tracker is tracking (TRACK_STAT_OK).
The value with index 0 represents the iris radius of the left eye. The value with index 1 represents the iris radius of the right eye. If iris is not detected, the value is set to -1.
An example of use:
| radius | Array that will be filled with iris radius values: index 0 represents iris radius of the left eye. Index 1 represents the iris radius of the right eye. |
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _getTrackerStatus | ( | int [] | tStatus | ) |
Returns tracker status.
An example of use:
| tStatus | Array that will be filled with tracking statuses for each of the tracked faces. |
| float _getTrackingQuality | ( | int | faceIndex | ) |
Estimated tracking quality for the current frame for the given faceIndex.
| faceIndex | Value between 0 and MAX_FACES-1, used to access the data of a particular tracked face. |
| void _grabFrame | ( | ) |
Grabs current frame from the camera.
Frame from the camera is obtained using OSXInput library. This function needs to be called before each call of the _track() function.
| void _initializeLicense | ( | string | license | ) |
Initializes the license.
It is recommended to initialize the license in the Awake () function within the unity script.
| license | Relative path from the working directory to the directory that contains license file(s) (e.g. Application.streamingAssetsPath + "/" + licenseFileFolder). |
| void _initTracker | ( | string | config, |
| int | numFaces | ||
| ) |
Initialises the tracker with configuration file and a desired number of tracked faces.
Number of tracked faces is limited within native plugin library with MAX_FACES parameter to 20.
Note that in order for license to be registered, _initializeLicense() method has to be called before.
| config | Relative path from the working directory to the directory that contains tracker configuration file, including configuration file name (e.g. Application.streamingAssetsPath + "/" + ConfigFileEditor). |
| numFaces | The maximum number of faces that will be tracked (<= 20). |
| void _openCamera | ( | int | orientation, |
| int | deviceID, | ||
| int | width, | ||
| int | height, | ||
| int | isMirrored | ||
| ) |
Initializes camera using native OSXInput library with given orientation, camera width and height, deviceID and isMirrored parameter for possible frame mirroring.
Device orientation parameter can have the following 4 values:
| orientation | Physical orientation of device. |
| deviceID | ID of the camera device from Windows system. For the first camera device value is 0, for the second 1 an so on. |
| width | Width of the frame. |
| height | Height of the frame. |
| isMirrored | 1 or 0, if set to 1 frames will be mirrored. |
| void _releaseTracker | ( | ) |
Releases memory allocated by the tracker in the _initTracker() function.
This function is called after the tracker ceases to be used or before the reinitialization of the tracker.
| void _setIPD | ( | float | IPDvalue | ) |
Sets the inter pupillary distance.
Inter pupillary distance (IPD) is used by the tracker to estimate the distance of the face from the camera. By default, IPD is set to 0.065 (65 millimeters) which is considered average. If the actual IPD of the tracked person is known, this function can be used to set this IPD. As a result, the calculated distance from the camera will be accurate (as long as the camera focal lenght is also set correctly). This is important for applications that require accurate distance. For example, in Augmented Reality applications objects such as virtual eyeglasses can be rendered at appropriate distance and will thus appear in the image with real-life scale.
IMPORTANT NOTE: In case of multitracking, same IPD will be set for all tracked faces!
| IPDvalue | the inter pupillary distance (IPD) in meters. |
| void _setTrackerConfiguration | ( | string | trackerConfigFile, |
| bool | au_fitting_disabled, | ||
| bool | mesh_fitting_disabled | ||
| ) |
Sets tracking configuration.
The tracker configuration file name and other configuration parameters are set and will be used for the next tracking session (i.e. when _track() is called). Default configuration files (.cfg) are provided in Samples/data folder. Please refer to the VisageTracker Configuration Manual for further details on using the configuration files and all configurable options.
| trackerConfigFile | the name of the tracker configuration file. |
| au_fitting_disabled | disables the use of the 3D model used to estimate action units (au_fitting_model configuration parameter). If set to true, estimation of action units shall not be performed, and action units related data will not be available. Disabling will result in a small performance gain. |
| mesh_fitting_disabled | disables the use of the fine 3D mesh (mesh_fitting_model configuration parameter). If set to true, the 3D mesh shall not be fitted and the related information shall not be available. Disabling will result in a small performance gain. |
| void _track | ( | ) |
Performs face tracking on current frame.
Before each call of the _track() function, _grabFrame() function needs to be called to obtain the new frame.
1.8.15