arxiv:2609.15364

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

Published on Sep 14

· Submitted by

shicheng on Sep 15

AetherLabs-AI

Upvote

Authors:

Abstract

RSIAgent is a training-free multi-agent framework that enables recursive self-improvement via autonomous memory construction and broad-then-deep exploration to adapt digital agents to new environments.

Generated by thinkingmachines/Inkling-Small

Digital agents must often adapt to new environments whose interfaces, tools, and failure modes are not fully captured by pretrained models. We introduce RSIAgent, a training-free multi-agent framework for recursive self-improvement through autonomous memory construction. RSIAgent coordinates curriculum, actor, and verifier agents to continually explore the environment, validate outcomes, and retain environment-specific knowledge, including reusable causal relationships between actions, conditions, and consequences. It further adopts a broad-then-deep exploration strategy, combining parallel broad recursive self-exploration for discovering diverse environment structures with focused deep self-exploration for uncovering hard cases, hidden constraints, boundary conditions, and previously unknown causal dependencies. The resulting memory is frozen and can be directly reused for downstream tasks without updating model parameters. Experiments on OSWorld-v2 and Agent's Last Exam show that RSIAgent substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.

View arXiv page View PDF Project page GitHub 151 Add to collection

Community

Shichengf

Paper submitter about 14 hours ago

We’re excited to share RSIAgent: Autonomous Exploration for Recursive Self-Improvement in New Environments!

Instead of continuing to scale model parameters, we explore a different path: Scaling Experience. RSIAgent enables agents to autonomously decide what to explore, execute tasks, verify outcomes, and consolidate stable action–condition–outcome relationships into reusable memory, all while keeping the underlying model weights frozen.

RSIAgent coordinates curriculum, actor, and verifier agents through a Broad-to-Deep exploration strategy: broad exploration builds coverage across tools, workflows, and failure modes, while deep exploration targets difficult cases, hidden constraints, and boundary conditions.

When applied to Kimi-K3 and GLM-5.3, RSIAgent achieves:

78.98% Partial Score on OSWorld 2.0 (0808 offline), compared with 72.60% for GPT-6 Astra
84.82% on Agents’ Last Exam (Near-term), compared with 82.26% for GPT-6 Astra

These results suggest that agents can continue improving by autonomously acquiring, verifying, and reusing their own experience—even without updating model parameters.