Distributed Optimization of Modular Production Systems using Model-based Reinforcement Learning with Inverse Models

TL;DR

The paper introduces a model-based reinforcement learning framework that uses approximate inverse models to decouple actuation dynamics from state dynamics, enabling efficient policy training directly in task space. It employs a lightweight feedforward architecture for inverse models integrated into standard RL policies, improving training speed and performance on a modular production testbed. The method shows particular gains with off-policy algorithms in flexible manufacturing environments.

Why It Matters

Modular manufacturing systems require adaptive, scalable control strategies. This approach enables faster, more efficient self-learning control without relying on full system models, making it suitable for real-world industrial deployment where flexibility and rapid adaptation are critical.

Key Findings

• Integrating approximate inverse models allows RL policies to be trained exclusively in task space, simplifying learning and improving convergence.
• The lightweight feedforward inverse model architecture reduces computational overhead while maintaining high accuracy in dynamics approximation.
• Off-policy RL algorithms benefit significantly from the framework, achieving faster training and better performance on heterogeneous modular production systems.