|
visageSDK
|
VisageGazeTracker extends VisageTracker functionality, adding screen space gaze tracking on top of face tracking. More...
Public Member Functions | |||
| VisageGazeTracker (String trackerConfigFile) | |||
| |||
| ~VisageGazeTracker () | |||
| Destructor. More... | |||
| void | InitOnlineGazeCalibration () | ||
| Initializes online screen space gaze tracking. More... | |||
| void | AddGazeCalibrationPoint (float x, float y) | ||
| Passes a calibration point to the tracker in online screen space gaze tracking mode. More... | |||
| void | FinalizeOnlineGazeCalibration () | ||
| Finalizes online screen space gaze tracking calibration. More... | |||
| void | InitOfflineGazeCalibration (ScreenSpaceGazeRepository calibrator) | ||
| Initializes offline screen space gaze tracking. More... | |||
| float | GetGazeEstimations (ref ScreenSpaceGazeRepository repository) | ||
| Returns screen space gaze estimation data obtained in offline gaze tracking mode More... | |||
Public Member Functions inherited from VisageCSWrapper.VisageTracker | |||
| VisageTracker (String trackerConfigFile) | |||
| Initializes tracker with specified configuration file. More... | |||
| int[] | Track (int frameWidth, int frameHeight, Byte[] imageData, FaceData[] faceData, int format=VISAGE_FORMAT.RGB, int origin=VISAGE_ORIGIN.TL, int widthStep=0, long timeStamp=-1, int maxFaces=-1) | ||
| Performs face tracking (one or more faces) in the given image and returns tracking results and status. More... | |||
| void | SetTrackerConfiguration (String trackerConfigFile, bool au_fitting_disabled=false, bool mesh_fitting_disabled=false) | ||
| Set configuration file name. More... | |||
| void | SetTrackerConfiguration (VisageConfiguration configuration) | ||
| Apply a new Tracker configuration. More... | |||
| VisageConfiguration VisageConfiguration | GetTrackerConfiguration () | ||
| Obtain a copy of the active Tracker configuration. More... | |||
| void | Stop () | ||
| void | Reset () | ||
| Reset tracking. More... | |||
VisageGazeTracker extends VisageTracker functionality, adding screen space gaze tracking on top of face tracking.
For information about using the facial/head tracking refer to VisageTracker.
Screen space gaze tracking estimates gaze position (the location on the screen where the user is looking) in normalized screen coordinates. Screen space gaze tracking works in two phases: calibration and estimation.
In the calibration phase, the system is calibrated for gaze estimation by passing the calibration data to the tracker. Calibration data consists of series of points displayed on screen. The user looks at the calibration point. During the calibration phase tracker collects calibration points and matching tracking data for each point. After all calibration points have been passed to the tracker, the tracker performs calibration of gaze tracking system and switches to estimation phase.
In the estimation phase the tracker estimates gaze location in screen space coordinates and returns the data as ScreenSpaceGazeData object for each frame.
Screen space gaze tracking works in two different modes. Online mode is used when tracking in real time from camera. Offline mode is used when tracking from video files. The key differences between the two modes are:
| ONLINE | OFFLINE |
| Each FaceData object populated by the track() function contains screen space gaze data for the current frame. | Gaze estimation is done as a post process after the tracking is finished for the whole video sequence and returned as ScreenSpaceGazeRepository object. FaceData objects returned by the tracker during tracking contain no screen space gaze data for current frame. |
| Calibration data is passed to the tracker as it is shown on screen. | Calibration data is passed as a ScreenSpaceGazeRepository object for the whole video sequence. |
| Online mode can be initialized at any time during tracking. | Offline mode has to be initialized before tracking is started. |
| The application is responsible for finalizing gaze tracking system calibration. | Calibration is finalized automatically. |
Online mode implies using the screen space gaze tracking when tracking from camera.
It is initialized by calling InitOnlineGazeCalibration() method. This method prepares the tracker for real time gaze tracking calibration. Each calibration point in normalized screen coordinates is passed to tracker by calling AddGazeCalibrationPoint(). It is expected that the point is displayed on the screen before calling the method and that the user looks at calibration points during the calibration. Application is responsible for reading or generating the calibration data, displaying it on screen and synchronization with the tracker. It is required to manually notify the tracker that calibration is finished (once all calibration points are used) by calling FinalizeOnlineGazeCalibration() method. Once this method is called the tracker performs calibration of screen space gaze tracking system using provided calibration data and tracking data collected during the calibration process.
After the system is calibrated the estimation phase starts. Estimations are returned as part of FaceData objects obtained by calling the track() method, specifically in FaceData::gazeData.
Offline mode implies using gaze tracking feature when tracking from video file. It is assumed that calibration points have been displayed to the user while recording the video of the user's face; that user actually looked at the calibration points; and that these calibration points have been stored, together with corresponding time stamps. Furthermore, it is assumed that calibration was performed during a part of the video sequence. Gaze estimation is performed on the remaining parts of the video sequence. The calibration parts of the video need not be contiguous.
Offline mode initialized by calling InitOfflineGazeCalibration() method. This method takes ScreenSpaceGazeRepository object as parameter. This object contains calibration data for the tracked video sequence. Each calibration point consists of x and y coordinates given in normalized screen coordinates and the index of the frame in which calibration point was displayed to the user. The tracker reads calibration points from the provided repository and collects tracking data from corresponding frames of the tracked video sequence. Once all calibration data from provided repository is used, the tracker automatically performs calibration of the gaze tracking system using provided calibration data and tracking data collected during the calibration process.
Offline mode must be initialized before tracking is started, otherwise some of the calibration frames may be discarded.
After the whole video file has been processed, tracking stops and gaze estimations are available as ScreenSpaceGazeRepository object, obtained by calling GetGazeEstimations() method. The returned repository contains ScreenSpaceGazeData object with screen space gaze position for each non-calibration frame of the tracked video sequence.
Gaze tracking quality is available in online and offline mode on both frame and session level. Quality is returned as a value from 0 to 1 where 0 is the worst while 1 is the best.
In online mode quality is returned as a part of FaceData object. The frame level quality in online mode is returned as a quality parameter in FaceData::gazeData object. The session level quality in online mode is returned as a gazeQuality parameter in FaceData object. In online mode all frames passed to the tracker are considered part of a session, meaning that the quality is recalculated for each new frame processed by the tracker.
In offline mode quality is obtained using getGazeEstimations method. The frame level quality in offline mode is returned as a quality parameter in each ScreenSpaceGazeData object in ScreenSpaceGazeRepository. The session level quality in offline mode is returned by GetGazeEstimations function.
Definition at line 75 of file VisageGazeTracker.cs.
| VisageCSWrapper.VisageGazeTracker.VisageGazeTracker | ( | String | trackerConfigFile | ) |
| trackerConfigFile | The name of the tracker configuration file .cfg; default configuration files are provided in Samples/data folder. |
Constructor must be called with the full path to the configuration file. The configuration file is in the same directory as other data files needed by the tracker. Because it is in the same directory its path can be parsed to set the path for the other data files. For further details see VisageTracker Configuration Manual.
| VisageCSWrapper.VisageGazeTracker.~VisageGazeTracker | ( | ) |
Destructor.
| void VisageCSWrapper.VisageGazeTracker.AddGazeCalibrationPoint | ( | float | x, |
| float | y | ||
| ) |
Passes a calibration point to the tracker in online screen space gaze tracking mode.
This method is used in online gaze tracking mode to pass the position of the currently displayed calibration point to the tracker. This method should be called once for each calibration point, after the calibration point is displayed on the screen. Position of the calibration point is in normalized screen coordinates. The origin of the coordinate system is in the upper left corner of the screen; the lower right corner has coordinates (1, 1).
NOTE: Application is responsible for synchronization between the frequency of passing calibration points to the tracker and the frequency at which the tracker processes video frames. If calibration points are passed faster than the tracker works, it may happen that two (or more) calibration points are passed while the tracker is processing a single video frame. In such case, if the difference in speed is large enough, it is possible that the tracking data for the processed frame does not match to the calibration point. This reduces the quality of calibration and, consequently, estimation.
| x | x coordinate of the calibration point in normalized screen coordinates. |
| y | y coordinate of the calibration point in normalized screen coordinates. |
| void VisageCSWrapper.VisageGazeTracker.FinalizeOnlineGazeCalibration | ( | ) |
Finalizes online screen space gaze tracking calibration.
This method should be called after all calibration data is displayed and passed to the tracker. After this method is called the tracker performs calibration of gaze tracking system using the provided calibration data and the tracking data collected during the calibration phase.
After the calibration is finished, screen space gaze position is obtained as a part of FaceData object passed to the track() function. The returned FaceData object contains gaze position stored in ScreenSpaceGazeData object, specifically in FaceData::GazeData.
| float VisageCSWrapper.VisageGazeTracker.GetGazeEstimations | ( | ref ScreenSpaceGazeRepository | repository | ) |
Returns screen space gaze estimation data obtained in offline gaze tracking mode
This function is used for obtaining estimation data in offline mode (in online mode estimation data is a part of the FaceData object passed to the track() function). It returns the repository containing screen space gaze data for each non - calibration frame of processed sequence.
The method returns an empty array if called during tracking.
Note that each time this method is called, the memory for returned ScreenSpaceGazeRepository object is deallocated and reallocated.
| repository | Repository to be filled with screen space gaze data for last tracked sequence. |
| void VisageCSWrapper.VisageGazeTracker.InitOfflineGazeCalibration | ( | ScreenSpaceGazeRepository | calibrator | ) |
Initializes offline screen space gaze tracking.
Offline mode is used when tracking from video file. This method must be called before tracking is started using VisageTracker::Track(). Calibration data is passed to the tracker as a ScreenSpaceGazeRepository object containing a number of calibration points, each consisting of the position of the calibration point in normalized screen coordinates and the frame index in which the calibration point was displayed to the user. The data for each calibration point is stored as a ScreenSpaceGazeData object.
During the calibration phase tracker reads calibration points from the provided ScreenSpaceGazeRepository and collects tracking data in corresponding frames of the tracked video sequence. The gaze tracking system is calibrated automatically after all calibration points from the provided repository are used. After the tracking of the whole video sequence is finished, the screen space gaze positions can be obtained by calling GetGazeEstimations() method.
| calibrator | Pointer to ScreenSpaceGazeRepository containing calibration data. |
| void VisageCSWrapper.VisageGazeTracker.InitOnlineGazeCalibration | ( | ) |
Initializes online screen space gaze tracking.
Online mode is used when tracking from camera. This method starts the calibration phase of screen space gaze tracking. In the calibration phase the application displays the calibration data on the screen and passes it to the tracker using AddGazeCalibrationPoint(). Application is responsible for finishing the calibration phase by calling FinalizeOnlineGazeCalibration().
1.8.10