Overview

Reinforcement learning applied to foundation model reasoning, focusing on which post-training results generalize across domains, how RL affects a model’s capacity for further learning, and how to allocate a fixed compute budget across the components of an RL run.


Themes

  • Post-training dynamics — measuring loss of plasticity during RL post-training using formal measures rather than proxy symptoms such as entropy collapse or degraded pass@k
  • Distillation and specialization — multi-teacher on-policy distillation across model families and scales, and decomposing a model into domain specialists for recombination
  • Uncertainty in the training signal — formulating RLVR as partially observed, where neither reasoning validity nor verifier correctness is observable to the model
  • Structured actions in agentic settings — tool call sequences as combinatorial action spaces, learning valid call structure from logs
  • Efficiency and oversight — reasoning length and monitorability as explicit training objectives


Relevant prior work

Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective · NeurIPS 2025

Cheng et al., incl. Killian. Guru, an RL reasoning corpus spanning math, code, science, logic, simulation, and tabular reasoning. Establishes that several accepted findings in RL for reasoning are domain-specific.

IsoCompute Playbook: Optimally Scaling Sampling Compute for LLM RL · ICML 2026

Cheng et al., incl. Killian. Scaling laws for allocating compute across parallel rollouts, problem batch size, and update steps. Parallel rollouts per problem are the primary driver of performance, improving solution quality on easy problems and coverage on hard ones.

K2-Think: A Parameter-Efficient Reasoning System · MBZUAI IFM Technical Report, 2025

Cheng et al., incl. Killian. A 32B reasoning system combining long chain-of-thought training with test-time computation, competitive with substantially larger models on math, code, and science.

From Reasoning Traces to Reusable Modules · ICML 2026

Kong et al., incl. Killian. Theoretical and empirical account of compositional generalization under RL: exploration provides the coverage needed to identify latent structure in reasoning traces, and training on compound traces generalizes better than on isolated modules.

K2-V2: A 360-Open, Reasoning-Enhanced LLM · MBZUAI IFM Technical Report, 2025

K2 Team, Institute of Foundation Models. A 360B-parameter reasoning-centric base model released with its full training history and data.
Also relevant: PALU on concision as a Lagrangian trade-off between length and accuracy, Behavior Cue Reasoning on monitorable reasoning, LARK on trajectory selection for distillation by student learnability, and SR²AM on self-regulated simulative planning.