HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering

TL;DR

HierDoc introduces a two-stage hierarchical framework for long-document visual question answering, where page selection and region extraction are modeled as sequential, independently optimized decisions. It uses policy networks with structured rewards to route evidence from pages to regions, improving accuracy by leveraging both global document context and fine-grained visual elements.

Why It Matters

This approach enhances the reliability of document understanding systems in real-world applications like legal, medical, and financial document analysis, where evidence is scattered across multiple pages and regions. By decoupling and optimizing page and region selection separately, it enables more accurate and scalable reasoning over long documents.

Key Findings

• HierDoc achieves state-of-the-art performance on long-document VQA benchmarks, improving LongDocURL by 16.87% relative to the best open-weight baseline.
• Adding region-level evidence to a page-only system boosts accuracy by 5.51% and F1 by 4.82%, demonstrating the value of fine-grained selection.
• The stage-wise GRPO optimization with granularity-specific rewards enables effective training of both page and region policies without requiring answer-aware supervision.