When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control

TL;DR

The paper presents Chain-of-Self-Questioning (CoSQ), a prompt-based method that enables large language models to assess their own confidence before committing to answers. It reduces incorrect answers by 32.1% relative while maintaining high accuracy and answering most questions, with performance improvements across all tested models and thresholds. The approach supports tunable abstention decisions when factual support is insufficient.

Why It Matters

This method enhances reliability in high-stakes applications where incorrect answers are more harmful than not answering. It offers a lightweight, prompt-only solution for risk control without requiring model retraining or fine-tuning. The ability to dynamically adjust abstention thresholds makes it adaptable across diverse deployment scenarios.

Key Findings

• Grounded-CoSQ at a threshold of 0.90 reduces the wrong-commitment rate by 32.1% relative to chain-of-thought prompting while improving accuracy from 86.9% to 89.7%.
• All three CoSQ variants—Grounded, Critical, and Adaptive—outperform the baseline across eleven model families and multiple thresholds, with coverage rates between 86.5% and 88.6%.
• Results are validated on both multiple-choice and open-form datasets, confirming that self-assessment enables reliable, tunable abstention decisions when information is insufficient.