Neel Varma, Andrew Rufail, Dipika Khullar, Vasu Sharma
This paper analyzes the functional roles of register and high-norm patch tokens in self-supervised Vision Transformers. The results show that register-token features are associated with high-level semantic concepts, while outlier-token features are associated with lower-level patterns. This provides evidence for token specialization in self-supervised ViTs.
Understanding the roles of internal tokens in Vision Transformers can improve their performance and interpretability. This knowledge can be used to design more efficient and effective architectures. It also has implications for applications such as image classification and object detection.
Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.