Hybrid Reasoning Agents

gist

Researchers at Stanford, Together AI, UIUC and Meta build agents that switch between latent reasoning and text inside one generation. A small residual adapter called HybridLink feeds last-layer hidden states back as input embeddings, so the model can think without decoding tokens yet still call tools in text. Across 11 benchmarks at 4B, 8B and 14B scales, the authors report a 10.4% average gain over text and latent baselines, 35.5% to 54.7% fewer tokens, and 1.8x to 2.6x faster inference.

ideas

• HybridLink bridges hidden states and embeddings. A two-layer residual MLP maps each hidden state to the next input inside latent spans, and the agent keeps a key-value working memory of text, latent thoughts and tool results across turns.
• Training runs in two stages. HybridSFT adds a self-conditioned second pass that feeds HybridLink's own predictions back in, which the authors say lifts the 14B model on AIME25 from 23.8 to 32.5. HybridRL then learns when to think in latent space and when to act in text.
• The gains are largest on agentic search. At 14B, HybridAgent averages 52.4 on GAIA and 49.5 on WebWalker, against 43.7 and 40.5 for ARPO, and its pass@64 lead over a text agent reaches 76.7 versus 66.7.
• Latent thoughts stay readable and causal. The authors decode latent steps into pseudo-text that scores high on readability and context specificity, and MATH500 accuracy at 14B falls from 88.2 to 44.6 when latents are swapped for random token embeddings.
• Text remains enough for oversight. The paper reports that the visible text still lets weak and strong monitors flag errors and localize the first mistake about as well as, or better than, matched text agents.

quotes