Summer internship

Summary: The idea for this mentorship project is to research how fractional-order backpropagation can impact learning convergence in Vision Transformers. Previous studies involving fractional-order derivatives in backpropagation have shown improvements in training and testing accuracy. Building on that information, it would be interesting to investigate whether we can improve classification metrics across one, two, or more datasets (depending on time constraints) by comparing them with results from the classical integer-order backpropagation approach, using the same starting points and architectures.

Prerequisites: Laptop, Python, Pytorch, neural networks, and math (calculus) knowledge

Summary: This internship focuses on automatic emotion recognition using audio‑visual data, combining facial video and speech signals within a deep learning framework. The student will develop and evaluate neural network architectures that jointly model visual expressions and vocal cues to infer emotional states. Key tasks include extracting temporally aligned audio and video features, experimenting with different fusion strategies, and analyzing how each modality contributes to recognition performance. The project emphasizes representation learning from multimodal temporal data, robustness to real‑world variability, and critical evaluation of model behavior.

Prerequisites: Working knowledge of Python. Basics of deep learning (CNNs, RNNs, or Transformers). Familiarity with PyTorch and audio feature extraction is a plus.

Summary: Many injuries require physical rehabilitation, often in a specialized center and with specific equipment. In order to reduce the burden, we aim to showcase how computer vision can be used to improve the recovery process and increase engagement, especially in kids, by showcasing the exercise as games.

Prerequisites: Fundamentals of computer vision, game development, and SaaS

Summary: Images are usually stored in a variety of formats, such as JPEG and PNG, with the goal of striking a balance between compression and fidelity. However, the usual approaches do not take into account the content of the image, only the patterns in bytes of the rendering. This can have detrimental results on a computer vision pipeline – either consuming too much storage or producing artifacts not originally present in the image. Gaussian splatting is a technique that represents images as 2D Gaussians in a content-adaptive way. Recent works showed that the compression level of images resulting from its application is similar to JPEG, while achieving a significantly higher signal-to-noise ratio. The goal of this internship would be to assess a Gaussian splatting algorithm called ImageGS as a part of an image processing pipeline, including as an input to a Transformer model.

Prerequisites: Well-versed in Python, familiarity with Pytorch and/or Huggingface transformers

Summary: AprilTags are fiducial markers widely used in robotics to distinguish objects of interest—such as robots, tools, or environmental landmarks—from the rest of a scene. Each tag encodes a unique ID. By leveraging the known physical dimensions of a tag, we can determine the precise 6D pose (the 3D position and orientation) of the camera relative to the tag. When multiple tags are distributed throughout a space, they can be used to construct a sparse map and track a camera’s trajectory in real-time.

Addressing Lens Distortion: Raw camera images often suffer from geometric degradation, most notably at the edges of the frame where straight lines appear curved. This lens distortion must be corrected through a formal camera calibration process before the camera is used for precision tasks. During calibration, the camera captures a target with known geometry (such as a checkerboard). The camera model parameters are then tuned to minimize the re-projection error, ensuring the digital image accurately reflects the physical world. Because the accuracy of AprilTag localization relies entirely on the assumption of known geometric properties, performing this calibration precisely is a critical task. Consequently, we must carefully evaluate and validate the resulting model for any residual errors.

Prerequisites: Intermediate to advanced understanding of ROS and experience in Python/C/C++ and Linux

Summary: Simultaneous Localization and Mapping (SLAM) has been a popular field of research for over two decades. In a typical scenario, a mobile robot equipped with a LiDAR sensor traverses an unknown environment. As the robot collects measurements, it incrementally constructs a map while simultaneously estimating its own position within that space. The ultimate objective is to generate an accurate map and a precise trajectory of the robot’s movement.

While many modern approaches prioritize achieving high scores on standard dataset benchmarks, they often treat computational efficiency as a secondary concern. In real-world applications, however, hardware costs often limit the available computational and memory resources. Furthermore, SLAM systems must meet strict real-time execution requirements. Algorithms that perform exceptionally well on high-end servers may fail to deliver the same results when ported to edge devices. Additionally, power consumption is a critical factor, as it directly dictates the operational autonomy of battery-powered platforms.

The goal of this internship is to evaluate SLAM implementations through the lens of hardware constraints. By deploying various algorithms on a resource-limited platform (such as the NVIDIA Jetson Orin Nano), we will benchmark their performance based on both SLAM accuracy and computational efficiency.

Prerequisites: Software engineering, Python, basic Linux

Summary: This internship addresses the visual understanding of vegetation traversability in natural environments using deep learning. The student will develop models that distinguish between vegetation that can be traversed and obstacles that impede movement, using image or sensor data from outdoor scenes. The work involves exploring feature representations, leveraging weak or partial supervision, and designing learning pipelines that scale to large, diverse datasets. The project emphasizes efficient dataset generation, model generalization, and practical deployment considerations for autonomous navigation systems.

Prerequisites: Python, PyTorch, deep learning fundamentals