Robust ISAC Object Tracking via Cross-Modal Supervision and Spatio-Temporal Skip-Transformer

초록

Robust object tracking in ISAC systems is challenging due to the inherent sparsity and severe impulsive noise of millimeter-wave (mmWave) radar signals. To address this, we propose a novel tracking framework that combines cross-modal supervision with a Spatio-Temporal Skip-Transformer (ST-Skipformer). An offline LiDAR-camera label generator produces high-fidelity ground-truth supervision, enabling the ST-Skipformer to learn precise spatial features from noisy radar inputs. Furthermore, the ST-Skipformer incorporates a Temporal Skip (TS) Block to effectively filter background clutter while recovering high-frequency spatial details via skip connections. Experimental results on the real-world DeepSense 6 G dataset demonstrate that the proposed method achieves an Average Displacement Error (ADE) of 0.0174 m, reducing localization error by approximately 98% compared to baselines, while attaining a near-perfect F1-score of 0.9967.

키워드