Benchmarking World Models for Continual Learning on Compositional Tasks

TL;DR

The paper introduces a compositional continual learning benchmark for robot manipulation world models, where tasks are built from reusable components across action and perception dimensions. It evaluates both standard and modular world models, showing that modularity improves knowledge reuse over forgetting, but no method fully resolves the challenge. The benchmark enables finer analysis of how different modalities affect continual learning performance.

Why It Matters

Continual learning is essential for real-world robotics, where agents must adapt to new tasks without losing prior knowledge. This benchmark isolates knowledge reuse from learning speed, offering a clearer path to designing robust, adaptable world models. It helps identify bottlenecks in multimodal learning and guides future model architectures.

Key Findings

• Compositional tasks enable isolation of knowledge reuse from new learning, allowing more precise evaluation of continual learning performance.
• Modular world models with reusable components outperform standard models in balancing retention and adaptation.
• Perception and action modalities each introduce distinct bottlenecks in knowledge reuse, highlighting the need for modality-aware design.