|
VisageTrackerUnity
|
The VisageTrackerUnity sample project demonstrates the integration of visage|SDK with Unity game engine and is aimed at developers starting to use face tracking, face analysis and face recognition functionalities of visage|SDK in Unity game engine and development tool.
The sample project shows how face tracking is used to put a virtual object (glasses) on the user's face in Unity, and how to access the full 3D face model of the face. Sample also demonstrates how to use face tracking results to obtain face analysis and face recognition results. Developers may use parts of this project to speed up their own development in Unity using visage|SDK or use this application as a starting point for own projects.
The application needs to be built from Unity project which is located in Samples/Unity/VisageTrackerUnityDemo. Detailed instructions for building and running the project are available in section Building the project. If you wish to integrate visage|SDK in your own Unity project, instructions can be found here.
The files related to VisageTrackerUnity sample project are located in the Samples/Unity/VisageTrackerUnityDemo subfolder of the visage|SDK folder.
The application was built and tested with version 2020.3.21f1.
To build the application follow these steps:
In order to try the sample application, you need to have a valid license key. More information on how to obtain license key and include it in the application can be found in Licensing section of the documentation.
Tracking will start automatically when a face is found in the camera frame. Available buttons are:
On top of the video frames is a 3D model of glasses or tiger model (depending on which effect is chosen) that is transformed with the information about rotation and translation from the tracker. It blends seamlessly with the background video.
The project consists of a main scene with the Unity’s GameObjects that provide different functionalities. A GameObject is the main building block of a scene in Unity. It consists of different Components and can parent other GameObjects. Manipulation of GameObjects is done with Scripts. It is recommended for developers to get familiar with Unity basics before continuing. The integration with visage|SDK is done through a plugin (VisageTrackerUnityPlugin) that wraps native code calls to visage|SDK and provides functions that can be called from Unity scripts.
The core of the example is the Tracker GameObject. It consists of different components attached to it. The main component of the Tracker GameObject is the Tracker script component that handles the communication with the tracker (starts it and gets results from it). The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well. The existing Tracker script properties of interest are as follows:
Note: Because of the different coordinate systems used by the visage|SDK face tracker and the Unity game engine mirroring around x-axis is applied to the relevant GameObjects.
Objects in the scene that are also in the Controllable Objects list are the Glasses Model GameObjects.
Analyser GameObject is used to obtain and display face analysis results. It relies on the functionality of the Tracker object, i.e. Tracker script component. The GameObject consists of different components attached to it. The main component of the Analyser GameObject is the Analyser script component that handles the communication with the face analyser (initializes it and gets results from it). The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well. The existing Analyser script properties of interest are as follows:
Recognition GameObject is used to obtain and display face recognition results. It relies on the functionality of the Tracker object, i.e. Tracker script component. The GameObject consists of different components attached to it. The main component of the Recognition GameObject is the Recognition script component that handles the communication with the face recognition (initializes it and gets results from it). The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well. The existing Recognition script properties of interest are as follows:
There are two cameras in the scene, one for 3D scene (Main Camera) and other for video display (Video Camera). The main camera "Field of View" is automatically set according to the Tracker focus (camera_focus parameter in configuration file) and input image aspect by the Tracker script. The Video Camera renders objects in the Video layer by using orthographic projection.
Video GameObject contains Video Plane that is used for displaying video and is automatically scaled to input frame aspect ratio by the VideoController script attached to it. In this project the material used by the Video Plane is tinted green to differentiate it from the textured 3D face model that uses the same texture.
The rotation and translation information from the tracker are applied to the Glasses object as they are in the Tracker script Controllable Objects list. This enables overlaying virtual objects to the tracked face that transform correctly with it. Other custom objects can be used instead of the glasses object. For the correct size and positioning of the glasses objects and also on how to achieve occlusion refer to the Animation & AR modeling guide.
The integration of visage|SDK face tracking with Unity is done through a plugin that wraps native code calls and provides functions that can be called from Unity scripts. The provided prebuilt plugin VisageTrackerUnityPlugin.dll can be used as-is. For more information about the plugin including it's source code, see the VisageTrackerUnityPlugin project and its documentation.
1.8.10