Case Study: UAV Detection Proof of Concept (PoC)
Summary
This proof of concept (PoC) explores real-time detection, tracking, and motion prediction of small UAVs using thermal imaging on embedded hardware.
This approach integrates long-wave infrared sensing, temporal tracking, and predictive modeling to address challenges such as small target size, low-resolution data, and intermittent visibility. Instead of single-frame analysis, the system continuously tracks UAVs over time, even in noisy and dynamic environments.
The objective is to demonstrate reliable real-time UAV detection and tracking at operational distances of 60 to 100 meters, enabling deployment in environments with limited power, computing resources, and low latency requirements.
Thermal UAV Detection: Challenges and System Requirements
There is a growing demand for reliable, passive detection systems capable of identifying and tracking small drones and their behavior in real time. Clients require a solution that operates effectively in both day and night conditions, maintains continuous situational awareness, and delivers consistent performance in real-world environments.
However, these requirements introduce several technical challenges.
1. Environmental Heat (Solar Loading)
During the day, solar radiation affects the thermal characteristics of different surfaces (roofs, roads, equipment.) They absorb the heat and re-radiate it in the long-wave infrared (LWIR) spectrum. The phenomenon, known as thermal occlusion, masks the UAVs because their thermal signature becomes overwhelmed by their immediate context. This creates false negatives. They can cause loss of tracking and reduce system reliability.
As a result, detection models must generalize across a wide range of thermal backgrounds, which remains a key limiting factor in the current system.
2. Small Object Size
At distances of 60 meters or more, UAVs occupy a very small number of pixels in the thermal image. In these conditions, the drone often appears very small.
While the detection model is capable of identifying drones at these distances, confidence scores decrease as object size diminishes. This is primarily due to the limited amount of visual (thermal) information available.
3. Limited Distance Accuracy
The current system lacks precise distance measurement. Reliance on a single thermal camera results in the following limitations:
- Depth information cannot be obtained directly
- Minor detection errors, such as inaccuracies in bounding box size, can cause significant variations in estimated distance
System-Level Impact: Missed Detections at Range
The system experiences more missed detections at longer ranges due to the combined effects of small object size, environmental noise, and reduced detection confidence. This issue results from the interaction of these factors rather than any single cause.
Missed detections are critical from both business and operational perspectives. Inconsistent UAV detection and tracking undermine situational awareness and reduce the system’s effectiveness in security, monitoring, and defense.
To address these challenges, the system should move beyond isolated frame-based detection and adopt a unified approach that integrates detection, tracking, and motion prediction. This will improve UAV detection at operational distances, support persistent tracking despite intermittent visibility, and reduce false negatives to maintain continuous target awareness.
Additionally, the system must operate in real time on edge hardware to enable practical deployment in field environments that require low latency and computational efficiency.
Our Approach
System Overview and Processing Pipeline
Our approach consists of predicting the trajectory of tracked objects, even across occlusions if possible. The system combines thermal sensing, deep-learning detection, and sequential tracking into a unified real-time pipeline for detecting and tracking small UAVs in thermal imagery. It uses FLIT (Fusion of L2 and IoU Tracker), the tracking-by-detection method used for tracking small objects in thermal-infrared images.
1. Image Acquisition (Thermal Sensing)
A long-wave infrared (LWIR) camera continuously captures thermal video. Unlike RGB sensors, it detects heat signatures, enabling operation in low-light and nighttime conditions.
At longer distances (over 100 meters), UAVs appear as only a few pixels, leading to:
- Low signal-to-noise ratio
- Significant background clutter
- Strong reliance on temporal consistency
2. Thermal Object Detection
Training Data
The detection model was trained using a combination of:
- Open-source UAV datasets
- Dual-spectrum (RGB-T) data (visible + thermal infrared)
The primary dataset used in this PoC is SUAVE-600 v3, containing thermal imagery for small UAV detection. During development, an annotation issue affecting a subset of Anti-UAV data was identified and corrected, resulting in a measurable improvement in detection performance without introducing additional data.
Detection Proceoss
The model:
- Identifies candidate heat signatures
- Outputs bounding boxes around potential objects
- Assigns a confidence score to each detection
Only detections above a 50% confidence threshold are passed to the tracking stage.
3. Sequential Tracking (Tracking-by-Detection)
Instead of treating frames independently, the system uses the tracking-by-detection approach to link detections across frames into continuous object tracks.
The tracker:
- Associates detections using spatial proximity and motion consistency
- Maintains persistent object identities (IDs)
- Handles intermittent detections and short occlusions
Tracks are initialized after consistent detections and terminated when objects leave the frame (reappearing objects receive new IDs).
4. Motion Estimation and Prediction
Tracking is driven by a Kalman filter combined with the Hungarian algorithm for optimal association.
For each UAV, the system estimates position, velocity, and direction, enabling prediction of future positions, smoothing of noisy detections, and maintaining track continuity during missed detections.
5. Camera Motion Compensation
To handle moving cameras (e.g., pan-tilt systems), global motion is estimated between frames to ensure stable and accurate tracking under dynamic conditions.
This separates:
- Apparent motion resulting from camera movement
- Actual movement of drones
Results and Outcomes
- Detection outputs with approximately 50% confidence become more reliable when combined with motion tracking, which reduces false negatives and maintains temporal stability.
- Enhancements to the detector, including improved annotation quality, augmentations, and loss functions, resulted in measurable gains in overall tracking performance.
- Using L2 distance in the cost matrix improves matching accuracy for fast and erratic UAV motion, which increases overall tracking performance (Higher Order Tracking Accuracy or HOTA).
Conclusion
This PoC shows that combining thermal detection, multi-frame tracking, and motion prediction with Kalman filtering significantly improves UAV detection reliability.
The system addresses key limitations of small-object thermal imagery and enables stable, real-time tracking at operational distances, supporting deployment in dynamic environments.
