DoubtLess: Training-Free Contrastive Decoding for Efficient Reasoning

TL;DR

DoubtLess is a training-free decoding method that uses a small auxiliary model to guide a larger reasoning model toward shorter, more efficient chains of thought. It compares the small model's responses to brief and verbose prompts and injects the difference into the large model's logits during generation. The approach reduces token usage by 31% on average without sacrificing accuracy across multiple reasoning benchmarks.

Why It Matters

It enables significant efficiency gains in reasoning tasks without requiring retraining or altering model architecture, making it applicable to existing models across domains. This improves deployment scalability and reduces computational costs for complex reasoning applications.

Key Findings

• DoubtLess reduces the length of chains of thought by 31% on average across seven reasoning benchmarks while maintaining original accuracy.
• Token savings primarily occur after the correct answer is first generated, indicating reduced redundant verification steps.
• The method generalizes across different model families and works synergistically with training-based compression techniques.