Semifactual Credit-Augmented Policy Optimization

Junshu Pan, Zhizhang Fu, Shulin Huang, Yiran Ding, Zifan Cheng, Wenqi Shao, Qiaosheng Zhang, Yue Zhang

arXiv: 2609.40360Published: September 30, 202636 upvotes

AI summary

TL;DR

The paper introduces Semifactual Credit-Augmented Policy Optimization (SCAPO), a method that improves reasoning and generalization in large language models by incorporating semifactual stability into token-level credit assignment. SCAPO outperforms existing methods on mathematics and out-of-distribution benchmarks. It achieves state-of-the-art results on Qwen3-4B-Base and Qwen3-1.7B-Base models.

Why It Matters

SCAPO's approach has practical significance as it enhances the reliability and accuracy of large language models, which is crucial for real-world applications. Improved reasoning and generalization capabilities can lead to better performance in tasks such as question-answering and decision-making. This can have a significant impact on industries that rely on AI-powered systems.

Key Findings

Substantial variation in token-level sensitivity to task-irrelevant prompt features
Suppressing high-drift token candidates improves reasoning accuracy
SCAPO outperforms GRPO on mathematics and out-of-distribution benchmarks

Topics

rlvr

llm

grpo

scapo

reinforcement learning

Thank you to arXiv for use of its open access interoperability. arxiv.org. aibrief.ai is not affiliated with or endorsed by arXiv.