LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation

Tianrun Yu, Kaixiang Zhao, Chih-Chun Chen, Amanda Hughes, Taylor W Killian, Fenglong Ma, Weitong Zhang, Porter Jenkins

NeurIPS 2026 arXiv PDF

• Selects teacher reasoning trajectories by how efficiently the student can learn them, with a χ²-regularized selection policy that preserves distributional coverage.

Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation

Kaixiang Zhao, Tianrun Yu, Shawn Huang, Porter Jenkins, Yushun Dong, Amanda Hughes

EMNLP 2026 arXiv PDF

• Shows that jointly conditioned self-correction can amplify hallucinations, and repairs high-risk facts via independently extracted observation and claim graphs.

Provable and Practical In-Context Policy Optimization for Self-Improvement

Tianrun Yu, Yuxiao Yang, Zhaoyang Wang, Kaixiang Zhao, Porter Jenkins, Xuchao Zhang, Chetan Bansal, Huaxiu Yao, Weitong Zhang

ICLR 2026 arXiv PDF

• Explains multi-round self-reflection as in-context policy optimization with provable guarantees, and proposes ME-ICPO for affordable test-time scaling on math reasoning.

Recovering Reliable Supervision via Critique-Guided On-Policy Distillation

Kaixiang Zhao, Tianrun Yu, Shangzhe Li, Yuxiao Yang, Amanda Hughes, Weitong Zhang, Taylor W Killian

arXiv 2026 arXiv PDF Website

• Shows that the OPD teacher often follows student errors even though it recognizes them, and restores reliable supervision (CriticOPD) by having the teacher critique incorrect rollouts and the student rewrite from the last error.

When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation

Yuxiao Yang, Tianrun Yu, Shangzhe Li, Kaixiang Zhao, Xuchao Zhang, Chetan Bansal, Huaxiu Yao, Taylor W Killian, Weitong Zhang

arXiv 2026 arXiv PDF Website 🤗 #3 Paper of the Day

• Identifies termination-token mismatch between student and teacher as a key source of length inflation in on-policy distillation, and mitigates it by treating equivalent EOS tokens as a shared stopping action.

Rethinking Training–Inference Mismatch in LLM Reinforcement Learning: Where It Arises and How to Correct It

Tianrun Yu, Kaixiang Zhao, Shangzhe Li, Yuxiao Yang, Porter Jenkins, Weitong Zhang, Taylor W Killian

arXiv 2026 arXiv PDF Website

• Characterizes the training–inference mismatch in RLVR as an additive log-odds displacement, and proposes calibrated importance sampling (CIS), a confidence-aware truncation with a bounded second moment.

REVO: Rollout-Efficient Off-Policy Distillation via Variance-Guided Reuse

Yuxiao Yang, Shangzhe Li, Tianrun Yu, Kaixiang Zhao, Taylor W Killian, Weitong Zhang

arXiv 2026 arXiv PDF

• Reuses each student rollout for multiple updates via one-step resampling, stabilized prefix weighting, and variance-guided token priorities; 50 rollout iterations match or exceed OPD trained for 200.

An RL View of OPD: Least Square Policy Distillation for Sample-Efficient LLM Reasoning

Shangzhe Li, Yuxiao Yang, Tianrun Yu, Kaixiang Zhao, Xiaoyun Wang, Taylor W Killian, Weitong Zhang

arXiv 2026 arXiv PDF

• Connects reverse-KL OPD to KL-regularized policy optimization, and brings optimistic exploration and off-policy data reuse into distillation to preserve policy diversity.