|
visageSDK
|
Objective-C wrapper of the VisageSDK::FaceData class. More...
#import <objc_FaceData.h>
Inherits NSObject, and <NSCopying>.
Properties | |
| float | trackingQuality |
| Tracking quality level. More... | |
| float | frameRate |
| The frame rate of the tracker, in frames per second, measured over last 10 frames. More... | |
| long | timeStamp |
| Timestamp of the current video frame. More... | |
| float * | faceTranslation |
| Translation of the head from the camera. More... | |
| float * | faceRotation |
| Rotation of the head. More... | |
| float * | faceRotationApparent |
| Rotation of the head from the camera viewpoint. More... | |
| float * | gazeDirection |
| Gaze direction. More... | |
| float * | gazeDirectionGlobal |
| Global gaze direction, taking into account both head pose and eye rotation. More... | |
| float * | eyeClosure |
| Discrete eye closure value. More... | |
| float * | irisRadius |
| Iris radius values in px. More... | |
| int | shapeUnitCount |
| Number of facial Shape Units. More... | |
| float * | shapeUnits |
| List of current values for facial Shape Units, one value for each shape unit. More... | |
| int | actionUnitCount |
| Number of facial Action Units. More... | |
| int * | actionUnitsUsed |
| Used facial Action Units. More... | |
| float * | actionUnits |
| List of current values for facial Action Units, one value for each Action Unit. More... | |
| const char ** | actionUnitsNames |
| List of facial Action Units names. More... | |
| FDP * | featurePoints3D |
| Facial feature points (global 3D coordinates). More... | |
| FDP * | featurePoints3DRelative |
| Facial feature points (3D coordinates relative to the face origin, placed at the center between eyes). More... | |
| FDP * | featurePoints2D |
| Facial feature points (2D coordinates). More... | |
| int | faceModelVertexCount |
| Number of vertices in the 3D face model. More... | |
| float * | faceModelVertices |
| List of vertex coordinates of the 3D face model. More... | |
| float * | faceModelVerticesProjected |
| List of projected (image space) vertex coordinates of the 3D face model. More... | |
| int | faceModelTriangleCount |
| Number of triangles in the 3D face model. More... | |
| int * | faceModelTriangles |
| Triangles list for the 3D face model. More... | |
| float * | faceModelTextureCoords |
| Texture coordinates for the 3D face model. More... | |
| float * | faceModelTextureCoordsStatic |
| Static texture coordinates of the mesh from the configuration file parameter mesh_fitting_model. More... | |
| int | faceScale |
| Scale of face bounding box expressed in pixels. More... | |
| VisageRect | faceBoundingBox |
| Face bounding box. More... | |
| float | cameraFocus |
| Focal distance of the camera, as configured in the tracker/detector configuration file. More... | |
| float | gazeQuality |
| The session level gaze tracking quality in online mode. More... | |
Objective-C wrapper of the VisageSDK::FaceData class.
Definition at line 29 of file objc_FaceData.h.
|
readatomicassign |
Number of facial Action Units.
This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). Face detector leaves this variable undefined.
Number of action units that are defined for current face model.
Definition at line 252 of file objc_FaceData.h.
|
readatomicassign |
List of current values for facial Action Units, one value for each Action Unit.
Action Units are the basic motions of the face, such as mouth opening, eyebrow raising etc. This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). Face detector leaves this variable undefined.
The Action Units used by the tracker are defined in the 3D face model file specified by au_fitting_model parameter in the configuration file (see the VisageTracker Configuration Manual for details). Furthermore, the tracker configuration file defines the names of Action Units and these names can be accessed through actionUnitsNames. Please refer to section or VisageTracker Configuration Manual for full list of Action Units.
Definition at line 277 of file objc_FaceData.h.
|
readatomicassign |
List of facial Action Units names.
This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). Face detector leaves this variable undefined.
Definition at line 286 of file objc_FaceData.h.
|
readatomicassign |
Used facial Action Units.
This variable is set only while tracker is tracking (TRACK_STAT_OK) and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details). Face detector leaves this variable undefined.
List of values, one for each action unit, to determine if specific action unit is actually used in the current tracker configuration. Values are as follows: 1 if action unit is used, 0 if action unit is not used.
Definition at line 263 of file objc_FaceData.h.
|
readatomicassign |
Focal distance of the camera, as configured in the tracker/detector configuration file.
This variable is set while tracker is running (any status other than TRACK_STAT_OFF), or if the detector has detected a face.
Focal length of a pinhole camera model used as approximation for the camera used to capture the video in which tracking is performed. The value is defined as distance from the camera (pinhole) to an imaginary projection plane where the smaller dimension of the projection plane is defined as 2, and the other dimension is defined by the input image aspect ratio. Thus, for example, for a landscape input image with aspect ratio of 1.33 the imaginary projection plane has height 2 and width 2.66.
This value is used for 3D scene set-up and accurate interpretation of tracking data.
Corresponding FoV (field of view) can be calculated as follows:
fov = 2 * atan( size / (2*cameraFocus) ), where size is 2 if width is larger than height and 2*height/width otherwise.
This member corresponds to the camera_focus parameter in the tracker/detector configuration file.
Definition at line 528 of file objc_FaceData.h.
|
readatomicassign |
Discrete eye closure value.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
Index 0 represents closure of left eye. Index 1 represents closure of right eye. Value of 1 represents open eye. Value of 0 represents closed eye.
Definition at line 212 of file objc_FaceData.h.
|
readatomicassign |
Face bounding box.
The bounding box is rectangular determined by the x and y coordinates of the upper-left corner and the width and height of the rectangle. Values are expressed in pixels.
Definition at line 510 of file objc_FaceData.h.
|
readatomicassign |
Texture coordinates for the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
A pair of u, v coordinates for each vertex. When FaceData is obtained from the tracker, the texture image is the current video frame. When FaceData is obtained from detector, the texture image is the input image of the detector.
Definition at line 491 of file objc_FaceData.h.
|
readatomicassign |
Static texture coordinates of the mesh from the configuration file parameter mesh_fitting_model.
They can be used to apply textures based on the texture template for the unwrapped mesh to the face model. The texture template for these coordinates is provided in jk_300_textureTemplate.png This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
A pair of u, v coordinates for each vertex.
Definition at line 501 of file objc_FaceData.h.
|
readatomicassign |
Number of triangles in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
Definition at line 472 of file objc_FaceData.h.
|
readatomicassign |
Triangles list for the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK), or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
Each triangle is described by three indices into the list of vertices faceModelVertices (counter-clockwise convention is used for normals direction).
Definition at line 481 of file objc_FaceData.h.
|
readatomicassign |
Number of vertices in the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
Definition at line 432 of file objc_FaceData.h.
|
readatomicassign |
List of vertex coordinates of the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
The format of the list is x, y, z coordinate for each vertex.
The coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-axis points up and z-axis points into the face - see illustration below.
To transform the coordinates into the coordinate system of the camera, use faceTranslation and faceRotation.
If the value set for the camera focal length in the tracker/detector configuration file corresponds to the real camera used, the scale of the coordinates shall be in meters; otherwise the scale is not known.
Definition at line 453 of file objc_FaceData.h.
|
readatomicassign |
List of projected (image space) vertex coordinates of the 3D face model.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face, providing that the mesh_fitting_model parameter is set in the configuration file (see VisageTracker Configuration Manual for details).
The format of the list is x, y coordinate for each vertex. The 2D coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
Definition at line 464 of file objc_FaceData.h.
|
readatomicassign |
Rotation of the head.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
This is the estimated rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero rotation (values 0, 0, 0) corresponds to the face looking straight ahead along the camera axis. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image, see illustration below. The values are in radians.
Note: The order to properly apply these rotations is y-x-z.
Definition at line 139 of file objc_FaceData.h.
|
readatomicassign |
Rotation of the head from the camera viewpoint.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
This is the estimated apparent rotation of the head, in radians. Rotation is expressed with three values determining the rotations around the three axes x, y and z, in radians. This means that the values represent the pitch, yaw and roll of the head, respectively. The zero apparent rotation (values 0, 0, 0) corresponds to the face looking straight into the camera i.e. a frontal face. Positive values for pitch correspond to head turning down. Positive values for yaw correspond to head turning right in the input image. Positive values for roll correspond to head rolling to the left in the input image. The values are in radians.
Note: The order to properly apply these rotations is y-x-z.
Definition at line 156 of file objc_FaceData.h.
|
readatomicassign |
Scale of face bounding box expressed in pixels.
Definition at line 504 of file objc_FaceData.h.
|
readatomicassign |
Translation of the head from the camera.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
Translation is expressed with three coordinates x, y, z. The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer - see illustration below. The global origin (0,0,0) is placed at the camera. The reference point on the head is in the center between the eyes.
If the value set for the camera focal length in the tracker/detector configuration file corresponds to the real camera used, the returned coordinates shall be in meters; otherwise the scale of the translation values is not known, but the relative values are still correct (i.e. moving towards the camera results in smaller values of z coordinate).
Aligning 3D objects with the face
The translation, rotation and the camera focus value together form the 3D coordinate system of the head in its current position and they can be used to align 3D rendered objects with the head for AR or similar applications. This example code shows how to do this using OpenGL.
The relative facial feature coordinates (featurePoints3DRelative) can then be used to align rendered 3D objects to the specific features of the face, like putting virtual eyeglasses on the eyes. Samples projects demonstrate how to do this, including full source code.
Definition at line 90 of file objc_FaceData.h.
|
readatomicassign |
Facial feature points (2D coordinates).
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
The 2D feature point coordinates are normalised to image size so that the lower left corner of the image has coordinates 0,0 and upper right corner 1,1.
The feature points are identified according to the MPEG-4 standard (with extension for additional points), so each feature point is identified by its group and index. For example, the tip of the chin belongs to group 2 and its index is 1, so this point is identified as point 2.1. The identification of all feature points is illustrated in the following image:
Certain feature points, like the ones on the tongue and teeth, can not be reliably detected so they are not returned and their coordinates are always set to zero. These points are: 6.1, 6.2, 6.3, 6.4, 9.8, 9.9, 9.10, 9.11, 11.4, 11.5, 11.6.
Several other points are estimated, rather than accurately detected, due to their specific locations. These points are: 2.10, 2.11, 2.12, 2.13, 2.14, 5.1, 5.2, 5.3, 5.4, 7.1, 9.1, 9.2, 9.6, 9.7, 9.12, 9.13, 9.14, 11.1, 11.2, 11.3, 12.1.
Ears' points - group 10 (points 10.1 - 10.24) can be either set to zero, accurately detected or estimated:
Face contour - group 13 and group 15. Face contour is available in two versions: the visible contour (points 13.1 - 13.17) and the physical contour (points 15.1 - 15.17). For more details regarding face contour please refer to the documentation of FDP class.
Nose contour - group 14, points: 14.21, 14.22, 14.23, 14.24, 14.25.
The resulting feature point coordinates are returned in form of an FDP object. This is a container class used for storage of MPEG-4 feature points. It provides functions to access each feature point by its group and index and to read its coordinates. Note that FDP stores 3D points and in the case of 2D feature points only the x and y coordinates of each point are used.
Definition at line 425 of file objc_FaceData.h.
|
readatomicassign |
Facial feature points (global 3D coordinates).
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
The coordinate system is such that when looking towards the camera, the direction of x is to the left, y iz up, and z points towards the viewer. The global origin (0,0,0) is placed at the camera, see illustration.
If the value set for the camera focal length in the tracker/detector configuration file corresponds to the real camera used, the returned coordinates shall be in meters; otherwise the scale is not known, but the relative values are still correct (i.e. moving towards the camera results in smaller values of z coordinate).
The feature points are identified according to the MPEG-4 standard (with extension for additional points), so each feature point is identified by its group and index. For example, the tip of the chin belongs to group 2 and its index is 1, so this point is identified as point 2.1. The identification of all feature points is illustrated in the following image:
Certain feature points, like the ones on the tongue and teeth, can not be reliably detected so they are not returned and their coordinates are always set to zero. These points are: 6.1, 6.2, 6.3, 6.4, 9.8, 9.9, 9.10, 9.11, 11.4, 11.5, 11.6.
Several other points are estimated, rather than accurately detected, due to their specific locations. These points are: 2.10, 2.11, 2.12, 2.13, 2.14, 5.1, 5.2, 5.3, 5.4, 7.1, 9.1, 9.2, 9.6, 9.7, 9.12, 9.13, 9.14, 11.1, 11.2, 11.3, 12.1.
Ears' points - group 10 (points 10.1 - 10.24) can be either set to zero, accurately detected or estimated:
Face contour - group 13 and group 15. Face contour is available in two versions: the visible contour (points 13.1 - 13.17) and the physical contour (points 15.1 - 15.17). For more details regarding face contour please refer to the documentation of FDP class.
Nose contour - group 14, points: 14.21, 14.22, 14.23, 14.24, 14.25.
The resulting feature point coordinates are returned in form of an FDP object. This is a container class used for storage of MPEG-4 feature points. It provides functions to access each feature point by its group and index and to read its coordinates.
Definition at line 336 of file objc_FaceData.h.
|
readatomicassign |
Facial feature points (3D coordinates relative to the face origin, placed at the center between eyes).
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
The coordinates are in the local coordinate system of the face, with the origin (0,0,0) placed at the center between the eyes. The x-axis points laterally towards the side of the face, y-xis points up and z-axis points into the face - see illustration below.
The feature points are identified according to the MPEG-4 standard (with extension for additional points), so each feature point is identified by its group and index. For example, the tip of the chin belongs to group 2 and its index is 1, so this point is identified as point 2.1. The identification of all feature points is illustrated in the following image:
Certain feature points, like the ones on the tongue and teeth, can not be reliably detected so they are not returned and their coordinates are always set to zero. These points are: 6.1, 6.2, 6.3, 6.4, 9.8, 9.9, 9.10, 9.11, 11.4, 11.5, 11.6.
Several other points are estimated, rather than accurately detected, due to their specific locations. These points are: 2.10, 2.11, 2.12, 2.13, 2.14, 5.1, 5.2, 5.3, 5.4, 7.1, 9.1, 9.2, 9.6, 9.7, 9.12, 9.13, 9.14, 11.1, 11.2, 11.3, 12.1.
Ears' points - group 10 (points 10.1 - 10.24) can be either set to zero, accurately detected or estimated:
Face contour - group 13 and group 15. Face contour is available in two versions: the visible contour (points 13.1 - 13.17) and the physical contour (points 15.1 - 15.17). For more details regarding face contour please refer to the documentation of FDP class.
Nose contour - group 14, points: 14.21, 14.22, 14.23, 14.24, 14.25.
The resulting feature point coordinates are returned in form of an FDP object. This is a container class used for storage of MPEG-4 feature points. It provides functions to access each feature point by its group and index and to read its coordinates.
Definition at line 383 of file objc_FaceData.h.
|
readatomicassign |
The frame rate of the tracker, in frames per second, measured over last 10 frames.
This variable is set while tracker is running., i.e. while tracking status is not TRACK_STAT_OFF. Face detector leaves this variable undefined.
Definition at line 46 of file objc_FaceData.h.
|
readatomicassign |
Gaze direction.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
This is the current estimated gaze direction relative to the person's head. Direction is expressed with two values x and y, in radians. Values (0, 0) correspond to person looking straight. X is the horizontal rotation with positive values corresponding to person looking to his/her left. Y is the vertical rotation with positive values corresponding to person looking down.
Definition at line 168 of file objc_FaceData.h.
|
readatomicassign |
Global gaze direction, taking into account both head pose and eye rotation.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
This is the current estimated gaze direction relative to the camera axis. Direction is expressed with three values determining the rotations around the three axes x, y and z, i.e. pitch, yaw and roll. Values (0, 0, 0) correspond to the gaze direction parallel to the camera axis. Positive values for pitch correspond to gaze turning down. Positive values for yaw correspond to gaze turning right in the input image. Positive values for roll correspond to face rolling to the left in the input image, see illustration below.
The values are in radians.
The global gaze direction can be combined with eye locations to determine the line(s) of sight in the real-world coordinate system with the origin at the camera. To get eye positions use featurePoints3D and FDP::getFP() function, e.g.:
Definition at line 204 of file objc_FaceData.h.
|
readatomicassign |
The session level gaze tracking quality in online mode.
Quality is returned as a value from 0 to 1, where 0 is the worst and 1 is the best quality. The quality is 0 also when the gaze tracking is off or calibrating.
Definition at line 536 of file objc_FaceData.h.
|
readatomicassign |
Iris radius values in px.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
The value with index 0 represents the iris radius of the left eye. The value with index 1 represents the iris radius of the right eye. If iris is not detected, the value is set to -1.
Definition at line 220 of file objc_FaceData.h.
|
readatomicassign |
Number of facial Shape Units.
This variable is set while tracker is running., i.e. while tracking status is not TRACK_STAT_OFF or if the detector has detected a face and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details).
Number of shape units that are defined for current face model.
Definition at line 229 of file objc_FaceData.h.
|
readatomicassign |
List of current values for facial Shape Units, one value for each shape unit.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face and only if au_fitting_model parameter in the configuration file is set (see VisageTracker Configuration Manual for details)..
Shape units can be described as static parameters of the face that are specific for each individual (e.g. shape of the nose).
The shape units used by the tracker and detector are defined in the 3D face model file, specified by the au_fitting_model in the configuration file (see the VisageTracker Configuration Manual for details).
Definition at line 242 of file objc_FaceData.h.
|
readatomicassign |
Timestamp of the current video frame.
This variable is set while tracker is running., i.e. while tracking status is not TRACK_STAT_OFF. Face detector leaves this variable undefined.
It returns the value passed to timeStamp argument in VisageTracker::track() method if it is different than -1, otherwise it returns time, in milliseconds, measured from the moment when tracking started.
Definition at line 55 of file objc_FaceData.h.
|
readatomicassign |
Tracking quality level.
This variable is set only while tracker is tracking (TRACK_STAT_OK) or if the detector has detected a face.
Estimated tracking quality level for the current frame. The value is between 0 and 1.
Definition at line 41 of file objc_FaceData.h.
1.8.15