Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning

TL;DR

The paper introduces UMM-Reflection, a method that applies reinforcement learning to unified multimodal models for self-repair of image generations. This approach jointly learns reflection text and image generation over the whole loop, outperforming supervised fine-tuning and naive RL methods. UMM-Reflection achieves significant improvements on various benchmarks.

Why It Matters

The proposed method has practical significance as it enables more accurate and efficient image generation and editing, which can be applied to various applications such as image synthesis, editing, and manipulation. This can lead to improved performance in computer vision tasks and enhanced user experience. The ability to self-repair generations can also reduce the need for manual editing and verification.

Key Findings

• UMM-Reflection outperforms supervised fine-tuning by 12.05 points on BAGEL
• The method achieves significant gains on unseen benchmarks
• Joint learning of reflection text and image generation is crucial for effective self-repair