Time-plane projection network for efficient time-series image analysis

초록

This paper proposes the Time-Plane Projection Network (TP2Net), a computationally efficient system for time-series image analysis that compresses video sequences into fixed-size 2D projected images. Unlike conventional approaches such as Recurrent Neural Networks, 3D convolutions, or Transformers, where computational costs scale with input sequence length, TP2Net generates two Time-Shape images that encode horizontal and vertical motion trajectories by projecting the temporal axis onto orthogonal planes. These are combined with two Spatial-Shape images representing the starting and ending frames of the video. Each of the four images is processed through independent 2D Convolutional Neural Networks in a single pass, and the extracted features are subsequently merged to derive spatiotemporal results. This architecture maintains a constant computational cost regardless of the video length. A flexible framework is presented through the comparative analysis of two architectural configurations: a performance-focused symmetric architecture employing the same pre-trained backbone (ShuffleNetV2, MobileNetV3, RegNet, ConvNeXt) for all branches, and an efficiency-focused asymmetric architecture utilizing lightweight networks for temporal feature extraction. Experimental results demonstrate that the symmetric architecture consistently outperforms the asymmetric counterpart. With the ConvNeXt-Base backbone, TP2Net achieved peak accuracies of 92.37% on UCF101 and 69.06% on HMDB51 at 61.68 GFLOPs. The ShuffleNetV2-x2.0 backbone achieved competitive accuracies of 87.29% on UCF101 and 51.52% on HMDB51 with only 2.37 GFLOPs, enabling real-time inference at 224.7 FPS on GPU and 42.0 FPS on CPU. Compared to the Two-Stream ConvNet, TP2Net achieves comparable accuracy while being over 900 times more computationally efficient, establishing a highly effective operating point on the accuracy-efficiency curve.

키워드