The VisageTrackerUnityDemo project shows how face tracking is used to put virtual object (glasses) on multiple faces in Unity, how to access the full 3D face model of the face and how to obtain and display face analysis (gender, age and emotion estimation) and face recognition results. Developers may use parts of this project to speed up their own development in Unity using visage|SDK or use this application as a starting point for own projects.
The sample project demonstrates the integration of visage|SDK with Unity game engine and
is aimed at developers starting to use VisageTracker, VisageFaceAnalyser and VisageFaceRecogntion functionalities of visage|SDK in Unity game engine and development
tool.
The application needs to be built from Unity project which is located in Samples/Android/VisageTrackerUnityDemo. Detailed instructions for building and running the project are available in sections Building the project and Generating the Android application. If you wish to integrate visage|SDK in your own Unity project, instructions can be found here.
The application needs to be built with license key in order to work properly. More information on how to obtain license key and include it in the application can be found in Licensing section of the documentation.
The files related to Visage Tracker Unity Demo sample project are located in the Samples/Android/VisageTrackerUnityDemo
sub-folder of the visage|SDK folder.
The prerequisites to building and running the sample project are :
To generate Android application for this sample follow these steps:
In case you are experiencing issues while building or running the project, please check the following:
When a face is found in the camera frame, tracking and face analysis starts. On top of the video frames is
the textured 3D face model generated and animated by the tracker and a 3D model of glasses that is transformed with
the information about rotation and translation from the tracker. You can switch between the two by pressing the button in the upper right corner.
You can also switch between front and back facing camera and pause the application by pressing corresponding buttons.
In the top left corner is a button which enables face recognition feature. After a certain face appears in the frame for the first time, there is an initialization phase where a predefined number of frames is used to collect data for face recognition. After that, the face gets a label in the format Person[N] (Person0, Person1...,Person[N]). All identities collected during the run of the application will additionally be displayed in a list on the left of the screen.
A rectangle element which is displayed from chin down contains face analysis results (age, gender and emotion estimation).
The project consists of a main scene with the Unity’s GameObjects that provide different functionalities. A GameObject is the main building block of a scene in Unity. It consists of different Components and can parent other GameObjects. Manipulation of GameObjects is done with Scripts. It is recommended for developers to get familiar with Unity basics before continuing. The integration with visage|SDK is done through the VisageTrackerUnityPlugin.
VisageTrackerUnityDemo includes pre-build step which copies required visage|SDK libraries and data files inside VisageTrackerUnityDemo project. Specifically, libraries are copied to Assets/Plugins/Android and data files to Assets/StreamingAssets/Visage Tracker. This is implemented in script PreBuild.cs located in Assets/Editor and executed before build.
The core of the example is the Tracker GameObject. It consists of different components attached to it. The main
component of the Tracker GameObject is the Tracker script component that handles the communication with the tracker
(starts it and gets results from it). The behaviour of the script can be modified by changing its properties and the script
code. Other properties can be added as well. The existing Tracker script properties of interest are as follows:
Analyser GameObject is used to obtain and display face analysis results.
It relies on the functionality of the Tracker object, i.e. Tracker script component.
The GameObject consists of different components attached to it. The main
component of the Analyser GameObject is the Analyser script component that handles the communication with the face analyser
(initializes it and gets results from it). The behaviour of the script can be modified by changing its properties and the script
code. Other properties can be added as well. The existing Analyser script properties of interest are as follows:
Recognition GameObject is used to obtain and display face recognition results.
It relies on the functionality of the Tracker object, i.e. Tracker script component.
The GameObject consists of different components attached to it. The main
component of the Recognition GameObject is the Recognition script component that handles the communication with the face recognition
(initializes it and gets results from it). The behaviour of the script can be modified by changing its properties and the script
code. Other properties can be added as well. The existing Recognition script properties of interest are as follows:
There are two cameras in the scene, one for 3D scene (Main Camera) and other for video display (Video Camera). The main camera "Field of View" is automatically set according to the Tracker focus (camera_focus parameter in configuration file) and input image aspect by the Tracker script. The Video Camera renders objects in the Video layer by using orthographic projection.
Video GameObject contains Video Plane that is used for displaying video and is automatically scaled to input frame aspect ratio by the VideoController script attached to it.
The rotation and translation information from the tracker are applied to the Glasses object as they are in the Tracker script Controllable Objects list. This enables overlaying virtual objects to the tracked face that transform correctly with it. Other custom objects can be used instead of the glasses object. For a more comprehensive guide on how to achieve occlusion or on how to properly introduce other objects to the scene see AR modeling guide.
The integration of visage|SDK face tracking with Unity is done through a plugin that wraps native code calls and provides functions that can be called from Unity scripts. For more information about the plugin including it's source code, see the VisageTrackerUnityPlugin project and its documentation.
Accessing and handling Android camera is done through Java plugin. For more information about the plugin including it's source code, see the AndroidCameraPlugin project and its documentation.