Anti-UAV Visual Tracking System

Grading

Detection, then tracking

A detector looks at one frame at a time. On video of a small drone against sky, trees and buildings, that has two costs. The detector misses the drone in some frames, and in others its most confident box lands on something else in the scene. A tracker carries information from one frame to the next. It predicts where the drone should be, uses that prediction to ignore detections that are too far away, and keeps an estimate through frames where the detector finds nothing. When it loses the drone anyway, it has to find it again. In this assignment you build both halves on the DUT Anti-UAV benchmark (Zhao et al., 2022). You fine-tune YOLO11n on a subset of the benchmark’s detection images and measure its [email protected] on the 2,200 test images. You then implement a constant-velocity Kalman filter with filterpy, with Mahalanobis gating and a rule for missed detections, and you run it on the benchmark’s 20 tracking videos. The scoring code is given. It computes the success-plot AUC and the precision at 20 pixels for the detector alone and for the detector with the filter, so you can say whether the filter helps and by how much. Before you change anything, the notebook runs top to bottom and scores a baseline: a detector fine-tuned for 3 epochs, and a tracker that reports the detector’s most confident box in each frame. Your fine-tuned detector has to beat the 3-epoch detector. The tracker is graded on what you measure and explain. Compare the Kalman filter with the detector alone on every score and say where the difference comes from. A filter that ties or loses on some scores is a valid result if you can say why. Track loss is the one place with a fixed requirement. When your tracker loses the drone it must find it again, well enough that its AUC on video13, the hardest sequence, is at least that of the detector alone. Over all 20 sequences it must also report fewer boxes than the detector alone in frames where the drone is out of view. The Kalman filter lecture covers the predict and update equations.

What you hand in

• The notebook, with every cell output saved and every TODO completed.
• The output video of one sequence, with the Kalman box and its trajectory drawn on every frame. The notebook writes it as <sequence>_kalman.mp4.

What you will need

You need a GPU runtime. In Colab, open Runtime, select Change runtime type, and choose a T4 GPU before starting. The default settings keep the run small. The detector trains for 30 epochs on 2,000 of the 5,200 training images, picks its best epoch on 400 validation images, and is scored on all 2,200 test images. It then runs once over all 24,804 frames of the 20 tracking videos. A free Colab session can end early. The notebook saves the converted images to disk, so after a disconnect it skips the conversion as long as the runtime’s files are still there. Training and the detection pass on the tracking videos run again. The data comes from the aegean-ai/dut-anti-uav dataset on the Hugging Face Hub, a copy of the authors’ release. Both parts load as Parquet files: the 10,000 detection images, and the 24,804 frames of the tracking videos with a ground-truth box for each frame. Some detection images are frames of the tracking videos: 285 in the training split and 139 in the validation split. The notebook holds them out of training, so the detector has not seen the frames the tracker is scored on.

Assignment notebook

Submitting

Commit the notebook and the output video to your private clone of the class repo under assignments/assignment-3. Save every cell output before committing. A notebook without outputs is graded as though its cells never ran. Submit the complete GitHub URL for that directory to Canvas or Brightspace. Do not submit the repository root. The assignment submission guide explains the remaining requirements, including how to set up your clone, add the TAs as collaborators, and create the README required in each assignment folder.

---