MOT3D: Real-Time 3D Multi-Object Tracking
C++17 tracking-by-detection on nuScenes: an EKF with a CTRV motion model, hand-written Hungarian association, and official nuScenes tracking metrics.
The problem
A self-driving car must know not just what is around it, but which object is which from one moment to the next. A pedestrian who disappears behind a truck has to come back as the same pedestrian, or prediction and planning fail.
What I built
- Implemented a 6-state CTRV extended Kalman filter, with a constant-velocity model behind the same interface for comparison
- Wrote the Hungarian assignment solver from scratch, with per-class gating on Mahalanobis, 3D IoU and center-distance costs
- Built a tentative → confirmed → dead track lifecycle with per-class birth and death thresholds
- Wired the pipeline end to end: SE(3) transforms, a nuScenes submission writer, official AMOTA scoring, and a ROS 2 node for rviz2
Key decisions
Hand-written Hungarian solver instead of a library
Association is the core of the tracker, so owning it makes cost gating and profiling transparent.
ROS-free core library
The tracker is tested and benchmarked in plain C++; ROS 2 is a thin wrapper, so it ports to any stack.
Reproduce a published baseline before adding anything
Every later change is measured against a trusted number, and every reported figure traces to a command.
Progress
0 of 7 milestones- Skeleton, Docker and CI
- Data layer and transforms
- Tracker core with tests
- First AMOTA score
- Six-configuration ablation
- Latency profiling and optimization
- ROS 2 node and rviz2 visuals