visageSDK
Classes | Namespaces | Functions | Variables
AndroidWrapper.cpp File Reference

Implementation of simple interface around visage|SDK VisageTracker functionality. More...

#include <jni.h>
#include <EGL/egl.h>
#include <GLES/gl.h>
#include <vector>
#include <stdio.h>
#include <unistd.h>
#include "VisageTracker.h"
#include "VisageRendering.h"
#include "AndroidImageCapture.h"
#include "AndroidCameraCapture.h"
#include "VisageLivenessAction.h"
#include "VisageLivenessBrowRaise.h"
#include "VisageLivenessSmile.h"
#include "VisageLivenessBlink.h"
#include "LivenessGUI.h"
#include <random>
#include "LicenseString.h"
#include <android/log.h>

Go to the source code of this file.

Classes

struct  Action
 Structure that holds the action and its accompanying messages and images that will be displayed to user during action verification. More...
 

Namespaces

 VisageSDK
 Namespace.
 

Functions

void AlertCallback (const char *warningMessage)
 Callback method for license notification. More...
 
long getTimeNsec ()
 Simple timer function.
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteLogoImage (JNIEnv *env, jclass type, jbyteArray logo_, jint width, jint height)
 Creates VsImage object for logo received from TrackerActivity. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFaceContourImage (JNIEnv *env, jclass type, jstring path, jbyteArray faceContourImage_, jint width, jint height)
 Creates VsImage object for face countour image received from TrackerActivity. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFontImage (JNIEnv *env, jclass type, jbyteArray fontImage_, jint width, jint height)
 Creates VsImage object for font image received from TrackerActivity. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackerInit (JNIEnv *env, jobject obj, jstring pathToAssets, jstring configFilename)
 Method for initializing the tracker. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_PauseTracker (JNIEnv *env, jobject obj)
 Method for "pausing" tracking. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromImage (JNIEnv *env, jobject obj, jint width, jint height)
 Method for starting tracking in image. More...
 
int Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_setParameters (JNIEnv *env, jobject obj, jint width, jint height, jint orientation, jint flip)
 Method that sets frame parameters. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromCam (JNIEnv *env, jobject obj)
 Method for starting tracking from camera. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackerStop (JNIEnv *env, jobject obj)
 Stops the tracker and cleans memory.
 
void Java_com_visagetechnologies_visagetrackerdemo_ImageTrackerView_WriteFrameImage (JNIEnv *env, jobject obj, jbyteArray frame, jint width, jint height)
 Writes raw image data into VisageSDK::AndroidImageCapture object. More...
 
void Java_com_visagetechnologies_visagetrackerdemo_JavaCamTrackerView_WriteFrameCamera (JNIEnv *env, jobject obj, jbyteArray frame)
 Writes raw image data into VisageSDK::AndroidCameraCapture object. More...
 
bool Java_com_visagetechnologies_visagetrackerdemo_TrackerRenderer_displayTrackingStatus (JNIEnv *env, jobject obj, jint width, jint height)
 Method for displaying tracking results. More...
 
float Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetFps (JNIEnv *env, jobject obj)
 Method for getting frame rate information from the tracker. More...
 
int Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetTrackTime (JNIEnv *env, jobject obj)
 Method for getting frame rate information from the tracker. More...
 
JNIEXPORT jstring JNICALL Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetStatus (JNIEnv *env, jobject obj)
 Method for getting the tracking status information from tracker. More...
 

Variables

static const int MAX_ACTION_STATES = 5
 Maximal number of states that the actions will have. More...
 
static const int NUM_ACTIONS = 3
 Number of actions that will need to be verified in order to confirm liveness.
 
Action actions [NUM_ACTIONS]
 Array of actions that will need to be verified.
 
VisageLivenessAction * BrowRaiseAction
 VisageLivenessBrowRaise object. More...
 
VisageLivenessAction * SmileAction
 VisageLivenessSmile object. More...
 
VisageLivenessAction * BlinkAction
 VisageLivenessBlink object. More...
 
GLuint frameTexId = 0
 Texture ID for displaying frames from the tracker.
 
float xTexCoord
 Texture coordinates for displaying frames from the tracker.
 
float yTexCoord
 Texture coordinates for displaying frames from the tracker.
 
int xTexSize
 Size of the texture for displaying frames from the tracker.
 
int yTexSize
 Size of the texture for displaying frames from the tracker.
 
float videoAspect
 Aspect of the video.
 
int glWidth
 Size of the OpenGL view.
 
int glHeight
 Size of the OpenGL view.
 

Detailed Description

Implementation of simple interface around visage|SDK VisageTracker functionality.

In order for Android application, which uses Java as its primary programming language, to communicate with visage|SDK functionality, which uses C++ as its primary language it is necessary to use Java Native Interface as a framework between the two.

Key members of wrapper are:

Definition in file AndroidWrapper.cpp.

Function Documentation

◆ AlertCallback()

void AlertCallback ( const char *  warningMessage)

Callback method for license notification.

Alerts the user that the license is not valid

Definition at line 221 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_ImageTrackerView_WriteFrameImage()

void Java_com_visagetechnologies_visagetrackerdemo_ImageTrackerView_WriteFrameImage ( JNIEnv *  env,
jobject  obj,
jbyteArray  frame,
jint  width,
jint  height 
)

Writes raw image data into VisageSDK::AndroidImageCapture object.

VisageTracker reads this image and performs tracking.

Parameters
framebyte array with image data
widthimage width
heightimage height

Definition at line 779 of file AndroidWrapper.cpp.

References VisageSDK::AndroidImageCapture::WriteFrame().

◆ Java_com_visagetechnologies_visagetrackerdemo_JavaCamTrackerView_WriteFrameCamera()

void Java_com_visagetechnologies_visagetrackerdemo_JavaCamTrackerView_WriteFrameCamera ( JNIEnv *  env,
jobject  obj,
jbyteArray  frame 
)

Writes raw image data into VisageSDK::AndroidCameraCapture object.

VisageTracker reads this image and performs tracking. User should call this function whenever new frame from camera is available. Data inside frame should be in Android NV21 (YUV420sp) format and VisageSDK::AndroidCameraCapture will perform conversion to RGB.

This function will reinitialize AndroidCameraCapture wrapper in case setParameter function was called, signaled by the orientationChanged flag. After creation, tracking will be resumed, signaled by trackerPaused flag.

Parameters
framebyte array with image data

Definition at line 798 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetFps()

float Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetFps ( JNIEnv *  env,
jobject  obj 
)

Method for getting frame rate information from the tracker.

Returns
float value of frame rate obtained from the tracker.

Definition at line 928 of file AndroidWrapper.cpp.

References VisageSDK::FaceData::frameRate.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetStatus()

JNIEXPORT jstring JNICALL Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetStatus ( JNIEnv *  env,
jobject  obj 
)

Method for getting the tracking status information from tracker.

Returns
tracking status information as string.

Definition at line 949 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetTrackTime()

int Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_GetTrackTime ( JNIEnv *  env,
jobject  obj 
)

Method for getting frame rate information from the tracker.

Returns
float value of frame rate obtained from the tracker.

Definition at line 938 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_PauseTracker()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_PauseTracker ( JNIEnv *  env,
jobject  obj 
)

Method for "pausing" tracking.

Causes the tracking thread to run without doing any work - blocks the calls to the track function

Definition at line 475 of file AndroidWrapper.cpp.

References NUM_ACTIONS.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_setParameters()

int Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_setParameters ( JNIEnv *  env,
jobject  obj,
jint  width,
jint  height,
jint  orientation,
jint  flip 
)

Method that sets frame parameters.

Called initially before tracking starts and every time orientation changes. Creates buffer of correct sizes and sets orientationChanged to true

Parameters
width- width of the received frame
height- height of the received frame
orientation- orientation of the frame derived from camera and screen orientation
flip- 1 if frame is mirrored, 0 if not

Definition at line 549 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackerInit()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackerInit ( JNIEnv *  env,
jobject  obj,
jstring  pathToAssets,
jstring  configFilename 
)

Method for initializing the tracker.

This method creates a new VisageTracker objects and initializes the tracker.

Parameters
configFilename- name of the configuration, along with the full path, to be used in tracking

Definition at line 412 of file AndroidWrapper.cpp.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromCam()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromCam ( JNIEnv *  env,
jobject  obj 
)

Method for starting tracking from camera.

Initiates a while loop. Image is grabbed and track() function is called every iteration. Copies data to the buffers for rendering.

Definition at line 588 of file AndroidWrapper.cpp.

References getTimeNsec(), VisageSDK::AndroidCameraCapture::GrabFrame(), VsImage::imageData, and VisageSDK::VisageTracker::track().

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromImage()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_TrackFromImage ( JNIEnv *  env,
jobject  obj,
jint  width,
jint  height 
)

Method for starting tracking in image.

Initiates a while loop. Image is grabbed and track() function is called every iteration. Copies data to the buffers for rendering.

Definition at line 496 of file AndroidWrapper.cpp.

References getTimeNsec(), VisageSDK::AndroidImageCapture::GrabFrame(), VsImage::imageData, and VisageSDK::VisageTracker::track().

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFaceContourImage()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFaceContourImage ( JNIEnv *  env,
jclass  type,
jstring  path,
jbyteArray  faceContourImage_,
jint  width,
jint  height 
)

Creates VsImage object for face countour image received from TrackerActivity.

Parameters
faceContourImage_- pixel data from loaded image
width- image width
height- image height

Definition at line 352 of file AndroidWrapper.cpp.

References VsImage::imageData, and VsImage::imageSize.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFontImage()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteFontImage ( JNIEnv *  env,
jclass  type,
jbyteArray  fontImage_,
jint  width,
jint  height 
)

Creates VsImage object for font image received from TrackerActivity.

Parameters
fontImage_- pixel data from loaded image
width- image width
height- image height

Definition at line 385 of file AndroidWrapper.cpp.

References VsImage::imageData, and VsImage::imageSize.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteLogoImage()

void Java_com_visagetechnologies_visagetrackerdemo_TrackerActivity_WriteLogoImage ( JNIEnv *  env,
jclass  type,
jbyteArray  logo_,
jint  width,
jint  height 
)

Creates VsImage object for logo received from TrackerActivity.

Parameters
logo_- pixel data from loaded logo
width- logo width
height- logo height

Definition at line 322 of file AndroidWrapper.cpp.

References VsImage::imageData, and VsImage::imageSize.

◆ Java_com_visagetechnologies_visagetrackerdemo_TrackerRenderer_displayTrackingStatus()

bool Java_com_visagetechnologies_visagetrackerdemo_TrackerRenderer_displayTrackingStatus ( JNIEnv *  env,
jobject  obj,
jint  width,
jint  height 
)

Method for displaying tracking results.

This method is periodically called by the application rendering thread to get and display tracking results. The results are retrieved using VisageSDK::TrackingData structure and displayed OpenGL ES for visual data (frames from camera and 3D face model). It shows how to properly interpret tracking data and setup the OpenGL scene to display 3D face model retrieved from the tracker correctly aligned to the video frame.

Parameters
widthwidth of GLSurfaceView used for rendering.
heightheight of GLSurfaceView used for rendering.

Definition at line 829 of file AndroidWrapper.cpp.

Variable Documentation

◆ BlinkAction

VisageLivenessAction* BlinkAction

VisageLivenessBlink object.

Used to verify that the user has blinked.

Definition at line 118 of file AndroidWrapper.cpp.

◆ BrowRaiseAction

VisageLivenessAction* BrowRaiseAction

VisageLivenessBrowRaise object.

Used to verify that the user has raised its eyebrows.

Definition at line 106 of file AndroidWrapper.cpp.

◆ MAX_ACTION_STATES

const int MAX_ACTION_STATES = 5
static

Maximal number of states that the actions will have.

It should be set to the number of states that the action with maximal number of states has.

Definition at line 81 of file AndroidWrapper.cpp.

◆ SmileAction

VisageLivenessAction* SmileAction

VisageLivenessSmile object.

Used to verify that the user has smiled.

Definition at line 112 of file AndroidWrapper.cpp.