Deep SORT: SIMPLE ONLINE AND REALTIME TRACKING WITH A DEEP ASSOCIATION METRIC

Date: 2017
Journal: CVPR

1. Introduction

SORT was simple framework that performs Kalman filtering in image space and frame by frame data association using the Hungarian method with an association metric that measures bounding box overlap

But it returns a relatively high # of identity switches as the employed association metric is only accurate when uncertainty is low

To overcome this issue by replacing association metric with a more informed metric that combines motion and appearance information

Deep SORT increase robustness against isses and occlusions while keeping the system easy to implement efficient and applicable to online

2. Sort with Deep Association Metric

2.1. Track Handling and State Estimation

The track handling and Kalman filtering framework is mostly identical to the original formulation

State space is defined $(u, ~v,~\gamma ,~h, ~\dot{x} ,~\dot{y}, ~\dot{\gamma},~\dot{h} )$

$\gamma$ aspect ratio

Tracks that exceed a predefined maximum age $A_{max}$ are considered to have left the scene and are deleted from the track set

2.2. Assignment Problem

To integrate motion and appearance information through combination of two appropriate metrics, Mahalanobis distance is used

$d^{(1)} (i, ~j) = (d_j - y_i ) ^T S_i ^{-1} (d_j - y_i )$

2.3. Matching Cascade

Mahalanobis distance favors large uncertainty because it effectively reduces the distance in standard deviations of any detection towards the projected track mean

It is an undesired behaviour as it can lead to increased track fragmentations and unstable tracks

In a final matching stage, intersection is done over union association as proposed in the original SORT algorithm

2.4. Deep Appearance Descriptor

A wide residual network with two convolutional layers followed by six residual blocks is employed