Polaris: A Gödel Agent Framework for Small Language Models through Experience-Abstracted Policy Repair

Paper: 2603.23129 Authors: Aditya Kakade, Vivek Srivastava, Shirish Karande (TCS Research) Categories: cs.LG

Abstract

Gödel agents realize recursive self-improvement: an agent inspects its own policy and traces, then modifies that policy in a tested loop. POLARIS introduces this capability for compact 7B models through experience-abstracted policy repair. Unlike response-level self-correction or parameter tuning, POLARIS makes policy-level changes with small, auditable code patches that persist and are reused on unseen instances. The agent engages in meta-reasoning: explaining errors, proposing concrete revisions, and updating its own policy. On MGSM, DROP, GPQA, and LitBench, a 7B model with POLARIS achieves consistent gains over the base policy and competitive baselines.

Key Contributions

• Experience abstraction: Distills failures into compact, reusable strategies that transfer to unseen instances
• Minimal code patch repair: Policy updates as small auditable patches, not opaque parameter changes
• Feasible for 7B models: Solves context growth and OOM issues that plagued Gödel Agent at small scale
• Persistent policy updates: Changes accumulate across instances within each benchmark, unlike per-response corrections

Why Gödel Agents Struggle at Small Scale

The original Gödel Agent framework retains multiple validation samples and prior evolution steps in memory for reflection. For 7B models, this causes:

• Context length explosion after each iteration
• Out-of-memory errors from growing computational overhead
• Tool-call failures before completing execution

POLARIS solves this by limiting retained failure examples and compressing tool-call history while preserving traceability.

The POLARIS Cycle

1. Recursive Self-Improvement Loop

1. Agent selects actions based on current policy and goals
2. Outputs and reasoning traces stored in memory
3. Evaluation collects failed tasks from validation set
4. Failed tasks trigger the Policy Repair module

2. Policy Repair via Experience Abstraction

1. Failure Analysis: Diagnose errors across N failed tasks
2. Strategy Synthesis: Abstract failures into reusable directives δ
3. Patch Generation: Convert directives into minimal code patches
4. Integration: Apply patches with runtime code mutation + execution checks
5. Conservative checks: Bounded retries ensure patches don’t break existing functionality

What Makes This Different

Approach	Scope	Persistence	Auditability
Self-Refine	Single response	No	Low
Parameter tuning	Model weights	Yes	Very low
Reflexion	Episode memory	Within episode	Medium
POLARIS	Policy code	Yes, across instances	High (code patches)

The key difference is that POLARIS changes are:

• Explicit: Concrete code patches you can read and review
• Persistent: Applied to all future instances, not just the current one
• Cumulative: Each repair builds on previous repairs
• Transferable: Strategies abstract away from specific failure instances

Results

Tested across diverse benchmarks:

• MGSM: Arithmetic reasoning
• DROP: Compositional inference
• GPQA: Graduate-level problem solving
• LitBench: Creative writing evaluation

A 7B model with POLARIS consistently outperforms the base policy and competitive baselines across all four benchmarks.

Takeaways

• Recursive self-improvement is possible for small models — the bottleneck was context management, not model capability
• Experience abstraction is the key enabler: compress failures into reusable strategies rather than retaining full trajectories
• Code patches as policy updates provide a rare combination of persistence, auditability, and transferability
• Meta-reasoning (agent explaining its own errors and proposing fixes) works even at 7B scale
• This opens a path toward self-improving agents that don’t require massive compute budgets