ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

TL;DR

ReToken introduces a single learnable token that acts as a retrieval target to select relevant visual tokens from a cached visual representation, improving performance on visual retrieval tasks without requiring full context processing. It achieves significant gains across image and video benchmarks with minimal computational overhead, enabling efficient long-context inference on a single H100 GPU.

Why It Matters

The method addresses scalability and performance issues in vision-language models when handling long visual sequences, making high-quality retrieval feasible in resource-constrained settings. Its zero-shot transferability to video tasks and lightweight design make it practical for real-world deployment.

Key Findings

• ReToken improves image retrieval performance by up to 13.4 points on Visual Haystacks and 12.4 points on LVBench benchmarks.
• The model achieves zero-shot transfer to long video retrieval with an 8.0-point gain on LVBench using Qwen3VL-8B.
• Training and inference fit within a single H100 GPU due to the model's minimal parameter count and efficient token selection mechanism.