HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis

TL;DR

HARMONY introduces a hierarchical reasoning framework that combines agentic visual language modeling with geometric point cloud refinement to reconstruct detailed 3D indoor scenes from single monocular images. It sequentially places objects using semantic reasoning and depth-first traversal, with feedback loops to correct errors and improve alignment with the input image.

Why It Matters

This approach enables more accurate and semantically coherent 3D scene reconstructions than prior methods, which struggle with either spatial consistency or geometric fidelity. It advances applications in virtual reality, robotics, and augmented reality by enabling richer scene understanding from a single image.

Key Findings

• HARMONY achieves better semantic consistency and perceptual alignment with input images compared to existing reconstruction baselines.
• The hierarchical object placement strategy, guided by agentic reasoning and depth-first traversal, reduces error propagation during scene construction.
• Geometry-based refinement using point cloud estimations improves the visual fidelity of rendered scenes, enhancing alignment with the original image.