Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning

TL;DR

The paper presents TRACE, a fast and effective attack that reconstructs private trajectory sequences from policy gradients in embodied reinforcement learning. It leverages temporal correlations and exact action recovery from gradient structure to outperform prior methods in reconstruction quality and speed.

Why It Matters

This work reveals a critical privacy vulnerability in distributed RL systems, showing that temporal gradient data can leak detailed behavioral sequences even when raw observations are kept on-device. It underscores the need for new privacy-preserving mechanisms that account for sequence-level information in gradient streams.

Key Findings

• TRACE achieves high-fidelity reconstruction of observation-action trajectories using only per-step policy gradients, with 18.8 dB PSNR and near-perfect action recovery.
• The attack exploits cross-time gradient correlations formalized via conditional mutual information and enables exact action recovery under small entropy regularization.
• TRACE is efficient (3–4.5 ms per frame), generalizable across architectures like transformers and recurrent networks, and effective on multimodal inputs and large action spaces.