Synthetic data offers many useful features when it comes to training machine learning models. However, a drawback is the possibility of generating unrealistic samples, which can hinder the model’s ability to transfer learned knowledge to real-world scenarios. The goal of this internship is to address this issue by applying a process that degrades the quality of an existing render of a person (synthetic image) and then using a generative model to restore the image. As a results, we hope to obtain a realistic image of a person, while still being able to reuse the original annotations such as face keypoints and bounding boxes.

Prerequisites: Python, Basic knowledge of deep learning

Photorealistic human face rendering will be investigated to generate synthetic face data. Data generation should be implemented using Unity with parametrization of head position, eye gaze, and facial expression. A final goal is to check if synthetic face data could be used in workflow for testing face tracking software. Known parameters and annotations in the face data would be compared to those predicted by the visage|SDK.

Prerequisites: Unity basics, C#

As we want our CNN network to perform well on diverse number of scenarios, we need data. As collection and annotation is hard and expensive, many scenarios are not covered by it. To get around this we could generate very specific scenarios to train our CNNs. One way to generate these specific scenarios would be using SESAME network, which can translate semantic maps, which carry pixel-wise information, to realistic looking images. The purpose of this internship is to implement such a network and examine how realistic are the car driving scenarios generated using this network.

Prerequisites: Deep Learning basics, Neural Networks, Python

The goal of this task is to compare sigma point methods like unscented Kalman filter (UKF) and cubature Kalman filter (CKF). The idea is to compare different methods in visual tracking use case as it would be interesting to see pros and cons of choosing sigma points differently. After understanding difference between UKF and CKF, implementation of square root variants to get numerical stability will be examined. To sum up, this task would first include implementation of UKF and then modifying it to CKF. Afterwards, implementing their square root variants and running comparison simulations. Finally, if time permits, implementation of other sigma points can be examined.

Prerequisites: Python (C/C++ is also acceptable), basic knowledge of tracking and Kalman Filter

Adverse weather conditions degrade image quality which causes our convolutional neural network to produce erroneous information that is relayed to other car subsystems. In order to properly evaluate the existing prediction quality model in these atypical scenarios, we need to have a way to get more soiling data than we already have. One approach that is considered in the modern automotive industry is to generate synthetic camera soiling scenarios out of existing footage.

Diffusion models are a relatively new approach to image generation whose performance appears to be comparable to the already established GAN model family, while possessing qualitative properties which make training and evaluation easier. Notable examples include OpenAI’s Dall-E 2, Stability AI’s Stable Diffusion, and Google’s Imagen.

The purpose of this internship is to get familiar with the diffusion model family, data preprocessing, data augmentation and model design. The main goal is to utilize a diffusion model for generating new soiled images using publicly available data. During this period, candidates will learn how to train diffusion models, evaluate them quantitatively and qualitatively and compare them with existing generative approaches in the literature.

Prerequisites: Python, Tensorflow, Deep Learning Basics, Neural Networks