Designer-RSI: Evolving Procedural Memory from User Traffic for Agentic Graphic Design

TL;DR

The paper presents Designer-RSI, a framework that enhances an AI agent's graphic design capabilities by maintaining a procedural memory of natural-language skills derived from real user interactions. This memory evolves through widening—adding new procedures for recurring tasks—and deepening—refining existing procedures based on execution outcomes—without updating the agent's core model. The approach significantly improves task success rates and quality across multiple benchmarks.

Why It Matters

It demonstrates a practical method for continual learning in real-world creative tasks where feedback is noisy and outcomes lack clear correctness signals. This enables AI agents to improve over time without human labeling or model retraining, making them more adaptable and reliable in professional design workflows.

Key Findings

• Procedural memory growth via widening and deepening mechanisms increased execution success on GenEval2 from 72.7% to 99.3% without any model weight updates.
• Combining widening and deepening led to a 58.5% win rate over a baseline agent on held-out user briefs, significantly outperforming either mechanism alone.
• The system achieved strong performance across specialized design benchmarks, showing robustness and generalization in unstructured, long-horizon tasks.