AIAny

Recursive Agent Optimization

Trains one model to delegate work recursively to copies of itself. Joint reinforcement learning teaches task splitting and communication, improving harder-task generalization while trading extra computation for parallel execution.

Introduction

Adding more agents only helps when they learn what to delegate. The central insight is to train the shared policy across the entire execution tree, so decomposition becomes a learned decision rather than a fixed orchestration rule.

Key Findings
  • On hard TextCraft-Synth tasks with an 8K context window, success rises from 0% for the single-agent baseline to 88% for the recursive agent.
  • With sufficient context, hard-task success improves from 20% to 88%; elapsed time on jointly solved tasks falls from 180.0 to 73.3 seconds, despite more environment steps.
  • DeepDive success improves from 24% to 40% on 50 held-out tasks, but execution becomes much slower. Parallel speedups depend on the task structure.
Who Should Read It

Useful for researchers training agents to split long or difficult problems. The evidence spans crafting, long documents, and web research; it does not establish universal latency or cost savings. Subtask evaluation may require an LLM judge, and recursion adds computation.

Why Train the Whole Tree?

Each node uses the same model. Local success signals, an optional child-success bonus, and depth balancing give intermediate work a training role instead of judging only the final answer.

Information

  • Websitearxiv.org
  • OrganizationsCarnegie Mellon University, Amazon AGI Labs
  • AuthorsApurva Gandhi, Satyaki Chakraborty, Xiangjun Wang, Aviral Kumar, Graham Neubig
  • Published date2026/05/07

More Items

Embeds an agent's reasoning together with its search query to improve retrieval. Trains a 4B embedding model on synthesized deep-research trajectories, helping preserve intent across multi-step searches.

Trains an 8B model to search, synthesize evidence, and write cited research reports. Its reinforcement-learning rewards use evolving rubrics grounded in retrieved evidence and contrasting model responses.

Trains long-horizon web researchers by combining agentic mid-training, synthetic trajectories, and reinforcement learning. A 30.5B-parameter model activates 3.3B parameters per token and learns to plan, search, and synthesize evidence.