Summer internship

This internship focuses on developing a neural network for improving color accuracy in images by leveraging camera metadata. Modern cameras automatically adjust settings like ISO, exposure and white balance, often leading to color inconsistencies. The project involves capturing a controlled dataset of color samples under different conditions, using this data to train a model that can predict more accurate colors based on both image content and metadata.

Prerequisites: Working knowledge of Python, deep learning knowledge, and knowledge of PyTorch/Tensorflow. Familiarity with image processing and color spaces is a bonus.

Skin tone estimation and foundation matching is one of the pillars of our Arbelle beauty offering. Accurately estimating the user’s skin tone is the starting point for our system to provide good recommendations. However, this is a hard problem to solve. In general, irregularities in illumination make different parts of the face to have lighter or darker shades and generally, averaging the values doesn’t provide the best accuracy. A potential solution is to add a “confidence value” to the samples, depending on how well they match the skin tone scale defined by Professor Monk (Monk Skin Tone scale or MST scale) and use different statistical models to make an accurate prediction. Similarly, we can use psycovisuals to our advantage in the recommendation process, by exploiting the fact that the human eye doesn’t perceive differences in the 3 RGB channels in the same way.

This proposal is to investigate what will be the best sampling and grading system for the skin tone and, given a minimum accuracy threshold, the best system to match the foundation products that we’ll recommend.

Prerequisites: Knowledge of C++, notions of computer vision (OpenCV) and color theory.

Models of today require a lot of data to work as intended. In order to fulfill the need for data, efficient process of selecting data for annotation is crucial.

The goal of this internship is to create a better way for selection of data by using convolutional autoencoders to extract vectors of features. These vectors of features will be used to find specific scenarios and similar data. By using techniques such as clustering and dimensionality reduction (PCA, t-SNE, UMAP) we will try to gain valuable insight into structure of data and which data to use to train models more efficiently. By combining these techniques and insights about data with active learning, we can boost development and training of downstream models.

Prerequisites: Python, convolutional neural networks, traditional machine learning (PCA, t-SNE), PyTorch/Tensorflow, knowledge of (convolutional) autoencoders is bonus.

Traditional multiple object tracking methods typically involve two distinct steps: object detection and tracking. However, recent advancements suggest that these steps can be integrated into a single process using end-to-end tracking approaches based on Transformer architectures. While these newer methods may be more complex to learn and require larger datasets, they offer potential gains in accuracy and robustness. Additionally, traditional methods are simpler and well-established but can suffer from sequential processing inefficiencies and error propagation.

This internship will focus on evaluating the feasibility of end-to-end tracking approaches in real-time applications, specifically in the context of vehicle tracking. Students will implement an end-to-end deep learning tracking approach and optimize the model to run as fast as possible with minimal reduction in performance. The goal is to assess the practicality of deploying these advanced tracking methods in real-world, time-sensitive scenarios.

Prerequisites: Basics of Deep learning and Transformers architecture. Working knowledge of Python and PyTorch/Tensorflow. Knowledge of traditional tracking approaches (Kalman Filter) is a bonus.

It is well known that today’s Advanced Driver Assistance Systems (ADAS) use multiple sensors, such as cameras, lidars, and radars. While most object detection methods focus on camera images, deep learning models for radar have only gained attention in recent years. Automotive radar is more robust than other sensors in adverse weather and lighting conditions. It is also less expensive. However, radar has limitations, such as lower resolution and higher noise levels. Despite these drawbacks, studies have shown that deep learning algorithms can effectively perform tasks like object detection and pose estimation with radar.

In this internship, we aim to implement a radar-only detection model and evaluate the performance in different driving scenarios.

Prerequisites: Convolutional Neural Network (CNN) basics, Python, knowledge of PyTorch is a bonus.

In adverse weather conditions such as fog, snow, or rain, light scattering degrades the information provided by camera sensors, implicitly worsening the performance of the whole autonomous driving perception stack. To address this limitation of camera-only systems, we look to introduce an additional, complementary sensor that is less affected by challenging weather scenarios.

This internship will explore radar and camera sensor fusion in latent space to enhance perception performance. The study will compare real-time camera-only and camera-radar models under the same computational constraints to determine if the radar-camera system performs better in challenging weather. The benefit from the research would help guide us in building safer autonomous driving stacks that are robust to different weather conditions.

Prerequisites: Python, PyTorch or Tensorflow, CNNs or Transformers

Traditional multiple object tracking (MOT) approaches rely on sensor fusion, typically implemented through a state estimator such as the Kalman filter (KF). The choice of KF depends on the properties of the system model and the specific use case. However, selecting the appropriate version can be challenging and is often an iterative process.

The aim of this internship is to use a concept called measure of nonlinearity (MoN) to evaluate the suitability of models for object tracking with specific state estimators. Open-source MOT datasets and standard mathematical models will be analyzed and the nonlinearity of these models evaluated across different state-space subsets, creating a “nonlinearity distribution.” The goal is to identify high nonlinearity scenarios, their frequency, maximum nonlinearity, and potentially other useful properties. Sequences with varying linearity will be tested using a sensor fusion library to determine if MoN correlates with Kalman filter performance. These insights could streamline dataset analysis and simplify model and KF version selection.

Prerequisites: Working knowledge of Python. Knowledge of dynamical systems (both linear and nonlinear), linear algebra and Kalman filter. Knowledge of multiple object tracking is a bonus.

Understanding and interpreting ML models is crucial for building trust and ensuring their reliable application in real-world scenarios. Recently, research in field of Explainable AI (XAI) provided methods that could help both improve understanding of networks and their performance.

The goal of this internship is twofold. First, to apply existing methods from field of Explainable AI to explore classical convolutional architectures and possibly expand analysis on more modern architectural approaches. Second expected output is to create graphical interface for comparing different architectures where different networks can be compared and evaluated using methods from Explainable AI field.

Prerequisites: Working knowledge of Python. Knowledge of deep learning in general, familiarity with CNNs (knowledge about various CNN architectures is a plus). Familiarity with TensorFlow and/or PyTorch frameworks.