SuperIgor: Self-Guided Plan Extraction for Instruction Following

Paper: 2604.20601 Authors: Zoya Volovikova, Nikita Sorokin, et al. Categories: cs.AI, cs.LG

Problem

Existing instruction-following methods rely on predefined subtasks requiring manual dataset annotation. This limits generalization to unseen instructions.

SuperIgor Framework

Enables language model to generate and refine high-level plans through a self-learning mechanism:

1. Iterative Co-training: RL agent trained to follow generated plans
2. Plan Adaptation: LM modifies plans based on RL feedback and preferences
3. Joint Improvement: Feedback loop where both agent and planner improve

Key Innovation

No manual annotation needed! Plans emerge from:

• RL feedback on task success
• Preference signals from environment
• Self-critique and refinement

Results

In environments with rich dynamics and stochasticity:

• Adheres to instructions more strictly than baseline methods
• Strong generalization to previously unseen instructions

Why Self-Guided Works

• Plans provide abstract representation that’s more transferable
• RL feedback directly optimizes plan quality
• Reduces distribution gap between training and deployment

Takeaways

• Removing human-annotated subtask decomposition enables better generalization
• Self-learning plan generation is viable for complex instruction following
• Joint optimization of planning and execution is key