Image Classifiers are Efficient Self-Supervised Video Representation Learners

TL;DR

The paper introduces VideoMSN, a framework for self-supervised video representation learning using image Vision Transformers. It achieves state-of-the-art performance on several benchmarks while requiring fewer pretraining epochs. VideoMSN shows strong performance in low-shot classification.

Why It Matters

This approach enables efficient video representation learning without relying on heavy 3D architectures or reconstruction-based autoencoders. It has practical significance in applications where labeled video data is scarce. The transferability of the learned representations is also confirmed in low-shot classification scenarios.

Key Findings

• VideoMSN achieves state-of-the-art performance on Kinetics-400, UCF101, and HMDB51
• Requires up to 32x fewer pretraining epochs compared to prior methods
• Shows strong performance in low-shot classification