Summer internship

When taking photos, we always want to look our best. Digitally, we use different filters to mask our imperfections, which blur our face and deliver unrealistic or unsatisfactory results. The goal of this project is to implement an approach to smooth facial blemishes while maintaining the original quality of the face region and the underlying skin texture, which will be powered by visage|SDK.

As a result, we aim to generate images with removed acne, wrinkles and redness, which appear untouched due to novel methods conserving original texture data and hair-like features.

Prerequisites: Python, Git, Deep Learning basics

Real-time face alignment systems suffer from irregular small-scale oscillations of predicted facial landmarks. They are detrimental to downstream tasks and typically result in jitterish visual elements. To quantify such oscillations through high-frequencies of landmark trajectories, one needs to make design choices that are reflected in the final quantization metric. The goal of this project is to illustrate the effect of those choices in a way understandable to non-experts. We will develop a simple tool for visualizing the amount of jitter in a video and use it to fine-tune the quantization metric so that it aligns with human perception.

Prerequisites: Python, Git, Basic machine learning

Synthetic datasets are a valuable asset for training and testing machine learning models, because they have well-known labels, can be easily expanded or adjusted, and can complement existing datasets to improve diversity or balance of less represented content.

The goal of this internship is to implement automated generation of datasets using Blender and Python scripting for creation of diverse synthetic face data with various useful labels (eye color, skin color, face shape, etc…). As a final step of the internship, we will assess the quality of the generated dataset by comparing the known dataset labels with predictions obtained by Visage Technologies’ makeup|SDK.

Prerequisites: Python, Basic knowledge of Blender

The autonomy of cars and robots greatly relies on the methods of motion estimation, one of which is visual odometry (VO). Visual odometry estimates the movement of an autonomous system using only images from a carry-on camera. In some cases, VO outperforms the accuracy of methods based on much more costly sensors such as LiDAR. Since VO computes the relative transformation between the initial and final pose by stacking estimations from consecutive frames, it is expected to obtain an inverse transformation if the same implementation uses the sequence played backwards. With that in mind, combining the two sequences (forward and backward), we expect to end up exactly where we started. However, due to various errors the transformation is not perfect and there is always a difference between the start and end pose. In this internship, the plan is to implement such a technique of evaluating VO accuracy using a forward-backward sequence.

Prerequisites: ROS, C/C++/Python basics, odometry and calibration knowledge recommended, but not essential

Monocular depth estimation is a critical challenge in automotive deep learning applications. The task becomes particularly complex when dealing with monocular cameras, which are commonly used in the industry to reduce costs and address calibration issues. This internship aims to enhance real-time depth estimation by employing knowledge distillation. We’ll start with simpler architectures for the teacher network and gradually explore more complex ones, all while maintaining a single-frame-based student network.

Prerequisites: Convolutional Neural Network (CNN) basics, Python, PyTorch or Tensorflow

In recent years, a new type of vulnerability of deep learning-based systems has been found. The vulnerability is based on corrupting a small subset of the training dataset by adding a specific pattern to some of the input data and changing the labels of the corrupted data. In safety critical environments, such as autonomous driving, this can cause the model to fail when such a pattern is introduced at inference time. The goal of the internship is to create a backdoor attack on a network that performs pedestrian detection. This means adding corrupted data into an existing dataset so that the network trained on that dataset either does not detect pedestrians containing a specific pattern, or detects pedestrians where none exist, triggered by that specific pattern. For additional validation, this will also be tested on real world data captured as part of the internship.

Prerequisites: Pytorch/Tensorflow, basics of deep learning

Statistical analysis is an important step in evaluating and comparing computer vision algorithms in terms of their performance and suitability for the problem at hand. The goal of this internship is to develop a Python-based tool for supporting such analysis. Given annotated data and algorithm output, the tool should match detections with annotations and compute various statistics such as precision, recall and intersection over union (IoU). The tool should be developed following best practices in software engineering, ensuring that the code is properly formatted, statically analyzed, fully tested, and documented. This will offer interns valuable knowledge and practical experience in professional software development.

Prerequisites: Python, Git, Object-oriented programming

Masked image modeling has been a dominant method for pretraining vision transformers in the last few years. Its applicability for pretraining vision CNNs was limited because most methods assumed that the network architecture was able to inherently process irregularly masked input. Over the last year, several methods have surfaced that aim to make masked image modeling applicable for pretraining vision CNNs, either by modifying the masking strategy and the loss function or by using submanifold sparse convolution. The goal of this internship is to use masked image modeling as a pretraining task methods for vision CNNs and evaluate the performance on a downstream object detection task.

Prerequisites: Python, deep learning, object detection and CNNs, PyTorch/Tensorflow