DriveOSS: Dataset and Benchmark for 3D Anomaly Segmentation and Open-set Semantic Segmentation in Autonomous Driving

TL;DR

DriveOSS introduces the first real-world multimodal dataset and benchmark for 3D anomaly and open-set semantic segmentation in autonomous driving, featuring 12,661 synchronized frames across diverse weather conditions with in-distribution and out-of-distribution subsets. It enables training and evaluation of models on unseen objects, addressing the limitations of closed-set assumptions in real-world deployment.

Why It Matters

Real-world autonomous driving requires robust perception of unknown objects, yet prior datasets lack out-of-distribution scenarios. DriveOSS enables development of safer, more adaptive perception systems by providing realistic, diverse, and annotated data for open-set challenges.

Key Findings

• Current 3D perception models struggle significantly with detecting unseen objects in real-world conditions.
• The inclusion of diverse weather conditions (rain, snow, night) exacerbates the difficulty of anomaly detection.
• DriveOSS enables fair benchmarking of open-set segmentation methods, revealing critical gaps in existing approaches.