4 /*! \mainpage visage|SDK VisageTrackerUnity example project
5 *
6 *
7 * The VisageTrackerUnity sample project demonstrates the integration of visage|SDK with Unity game engine and is aimed at developers
8 * starting to use face tracking, face analysis and face recognition functionalities of visage|SDK in Unity game engine and development tool.
9 *
10 * The sample project shows how face tracking is used to put a virtual object (glasses) on the user's face in Unity, and how to access
11 * the full 3D face model of the face. Sample also demonstrates how to use face tracking results to obtain face analysis and face recognition results.
12 * Developers may use parts of this project to speed up their own development in Unity using visage|SDK
13 * or use this application as a starting point for own projects.
14 * <br>The application needs to be built from Unity project which is located in <a href="../../../../../../Samples/Unity/VisageTrackerUnityDemo"><em>Samples/Unity/VisageTrackerUnityDemo</em></a>.
15 * Detailed instructions for building and running the project are available in section <a href="#building">Building the project</a>.
16 * If you wish to integrate visage|SDK in your own Unity project, instructions can be found <a href="../../../../../../doc/OpenGL/unity_integration.html">here</a>.
17 *
18 *
19 * \section building Building the project
20 *
21 * The files related to VisageTrackerUnity sample project are located in the Samples/Unity/VisageTrackerUnityDemo subfolder of the visage|SDK folder.
22 *
23 * The application was built and tested with version 2020.3.21f1.
24 *
25 *
26 * \subsection generateApp Building the application
27 *
28 * To build the application follow these steps:
29 * -# Open VisageTrackerUnityDemo project (<i>Samples/Unity/VisageTrackerUnityDemo</i>) in Unity.
30 * * <b>NOTE:</b> Depending on the Unity version that is used, multiple error messages related to project dependencies might appear in the console.
31 * Unity offers several possible solutions for this issue and more information can be found <a href="https://docs.unity3d.com/Manual/upm-errors.html#Upgrade"><em>here</em></a>.
32 * -# Add license key to the application (more information can be found in <a href="../../../../../../doc/OpenGL/licensing.html"><em>Licensing</em></a> section of the documentation).<br/>
33 * -# Select and open the <b>Main</b> scene.
34 * Please make sure the Analyser.cs script is attached to the Anaylser object in the Main scene.
35 * -# Select <i>File</i>-><i>Build settings</i> and under Platform choose <b>PC and Mac Standalone</b> and set <b>Windows</b> as target platform. Check if the correct Architecture is set: \if WIN64_DOXY <i>x86_64</i>\else <i>x86</i>\endif.
36 * -# Press the <b>Build</b> button to generate the project, a dialog window will appear. In this window, locate and set a destination folder to Samples/Unity/VisageTrackerUnityDemo/Build (the Build folder must first be created).
37 * <br/>
38 * \section using Using the sample application
39 *
40 * <br/>
41 * In order to try the sample application, you need to have a valid license key. More information on how to obtain license key and include it in the application can be found in
42 * <a href="../../../../../../doc/OpenGL/licensing.html"><em>Licensing</em></a> section of the documentation.
43 * <br/><br/>
44 *
45 * Tracking will start automatically when a face is found in the camera frame. Available buttons are:
46 * - Play/Pause in the lower left corner (resumes/pauses tracking).
47 * - Switch camera in the lower right corner.
48 * - Switch effect in the upper right corner of the screen (switches between rendering glasses or a tiger texture)
49 * - Enable face recognition in upper left corner of the screen
50 *
51 * On top of the video frames is a 3D model of glasses or tiger model (depending on which effect is chosen) that is transformed with the information about rotation and translation from the tracker. It blends seamlessly with the background video.
56 * The project consists of a main scene with the Unity’s GameObjects that provide different functionalities. A GameObject is the main building block of a scene in Unity. It consists of different Components and can parent other GameObjects. Manipulation of GameObjects is done with Scripts. It is recommended for developers to get familiar with Unity basics before continuing.
57 * The integration with visage|SDK is done through a plugin (VisageTrackerUnityPlugin) that wraps native code calls to visage|SDK and provides functions that can be called from Unity scripts.
58 *
59 * \subsection trackerGO Tracker GameObject
60 *
61 * The core of the example is the Tracker GameObject. It consists of different components attached to it. The main component of the Tracker GameObject is the Tracker script component that handles the communication with the tracker (starts it and gets results from it). The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well.
62 * The existing Tracker script properties of interest are as follows:
63 * - Controllable Objects: list of objects that will be transformed automatically while the face tracking is performed
64 * - Config File: filename of the configuration file that will be used by the tracker
65 *
66 * <b>Note:</b> Because of the different coordinate systems used by the visage|SDK face tracker and the Unity game engine mirroring around x-axis is applied to the relevant GameObjects.
67 *
68 * Objects in the scene that are also in the Controllable Objects list are the Glasses Model GameObjects.
69 *
70 * \subsection analyserGO Analyser GameObject
71 *
72 * Analyser GameObject is used to obtain and display face analysis results. It relies on the functionality of the Tracker object, i.e. Tracker script component.
73 * The GameObject consists of different components attached to it. The main component of the Analyser GameObject is the Analyser script component that handles the communication with the face analyser
74 * (initializes it and gets results from it).
75 * The behaviour of the script can be modified by changing its properties and the script code. Other properties can be added as well. The existing Analyser script properties of interest are as follows:
76 * - Data Path Analysis: path to data files used to initialize face analysis.
77 * - Analysis Data Element: prefab attached to the Analyser GameObject which contains the components needed to display information about the person's age, gender and emotions.
78 * - Analysis list: list of Analysis Data Elements instances for each tracked face that will be transformed while the face tracking is performed.
82 * Recognition GameObject is used to obtain and display face recognition results.
83 * It relies on the functionality of the Tracker object, i.e. Tracker script component. The GameObject consists of different components attached to it.
84 * The main component of the Recognition GameObject is the Recognition script component that handles the communication with the face recognition (initializes it and gets results from it).
85 * The behaviour of the script can be modified by changing its properties and the script code.
86 * Other properties can be added as well. The existing Recognition script properties of interest are as follows:
87 * - Data R Folder: path to data files used to initialize face recognition.
88 * - Recognition Gallery: UI panel object used to hold all collected indentities.
89 * - Identity: prefab attached to the Recognition GameObject which contains text mesh used to display indentity of a particular person that will be transformed while face tracking is performed.
90 *
91 * \subsection cameras Cameras
92 *
93 * There are two cameras in the scene, one for 3D scene (Main Camera) and other for video display (Video Camera). The main camera "Field of View" is automatically set according to the Tracker focus (camera_focus parameter in configuration file) and input image aspect by the Tracker script. The Video Camera renders objects in the Video layer by using orthographic projection.
94 *
95 * \subsection videoObject Video object
96 *
97 * Video GameObject contains Video Plane that is used for displaying video and is automatically scaled to input frame aspect ratio by the VideoController script attached to it. In this project the material used by the Video Plane is tinted green to differentiate it from the textured 3D face model that uses the same texture.
98 *
99 * \subsection glassesObject Glasses object
100 *
101 * The rotation and translation information from the tracker are applied to the Glasses object as they are in the Tracker script Controllable Objects list. This enables overlaying virtual objects to the tracked face that transform correctly with it. Other custom objects can be used instead of the glasses object.
102 * For the correct size and positioning of the glasses objects and also on how to achieve occlusion refer to the <a href="../../../../../../doc/OpenGL/modeling_guide.pdf">Animation & AR modeling guide</a>.
103 *
104 *
105 * \subsection plugin VisageTrackerUnityPlugin
106 * The integration of visage|SDK face tracking with Unity is done through a plugin that wraps native code calls and provides functions that can be called from Unity scripts. The provided prebuilt plugin VisageTrackerUnityPlugin.dll can be used as-is.
107 * For more information about the plugin including it's source code, see the VisageTrackerUnityPlugin project and its documentation.