Investigating Spatiotemporal Redundancy in Video Transformer for Collision Anticipation

TL;DR

This paper investigates spatiotemporal redundancy in video transformers for collision anticipation, finding that temporal token merging and neuron pruning can reduce computation without degrading performance. The study achieves a 1.76x speedup in inference time. The approach establishes a new pathway for faster algorithms.

Why It Matters

The research has practical significance as it enables low-latency inference on mobile robots and contributes to lower-carbon computation in construction. Faster and more efficient algorithms can improve worker-equipment proximity monitoring and collision anticipation. This can lead to enhanced safety and reduced accidents in construction sites.

Key Findings

• temporal token merging reduces computation by 50%
• neuron pruning preserves performance with 50% of MLP units retained
• token redundancy is axis-specific with greater redundancy across time than space