Verb-ICL: Rethinking In-Context Learning for Structured Prediction

TL;DR

Verb-ICL introduces a selective annotation method for in-context learning in structured prediction tasks by using token-level coverage to pick representative examples and generating actionable feedback that encodes task-specific annotation rules. This feedback enhances demonstration quality, leading to consistent performance gains across multiple datasets and model sizes, even with increasing annotation budgets.

Why It Matters

Structured prediction tasks often fail under standard in-context learning due to complex output formats and task-specific conventions. Verb-ICL improves model performance in low-resource settings by embedding human-like annotation guidance directly into demonstrations, making it practical for real-world applications with limited labeled data.

Key Findings

• Token-level coverage improves example selection by capturing local semantic patterns essential for structured outputs.
• Generated error feedback significantly boosts performance by encoding task-specific annotation rules, generalizing beyond individual corrections.
• Verb-ICL outperforms existing baselines across six datasets and maintains gains as annotation budgets grow, regardless of the underlying selection method.