OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics

Paper: 2606.09826
Authors: Mingxian Lin, Shengju Qian, Yuqi Liu, Yi-Hua Huang, Yiyu Wang, Wei Huang, Yitang Li, Fan Zhang, Zeyu Hu, Lingting Zhu
Categories: cs.CV, cs.AI

The Gap

Existing game benchmarks for VLM agents (e.g., GameBench, MineDojo, NetHack) share three limitations. First, they report a single first-attempt score per (agent, game) pair — a “cold-start” snapshot that ignores how an agent might improve after seeing its own mistakes. Second, nearly all focus on single-agent solo play, leaving out the social dimensions (competition, cooperation) that drive much of modern game AI. Third, there is no unified protocol: commercial VLMs (GPT-4V, Gemini) use different action interfaces than open-weight VLMs (LLaVA, Qwen-VL) or specialized game policies (behavior cloning agents), making apples-to-apples comparison impossible.

OmniGameArena fills all three gaps by building everything on Unreal Engine 5 with a unified action interface (same joystick-like inputs for all agents), and by introducing the Improvement Dynamics Curve (IDC) — a multi-round reflection harness that reveals how agents learn. The logical spine:

[Problem: single-score, solo-only, no unified protocol ]
    |
    v
[Assumption: iterative reflection reveals learning ability ]
    |
    v
[Method: 12 UE5 games (7 solo, 3 PvP, 2 coop) + IDC ]
    |
    v
[Evidence: IDC curves + held-out variant scores for 12 agents ]
    |
    v
[Conclusion: IDC provides richer signal than cold-start score ]

The Increment

One sentence: Before this paper, you could only ask “how well does this agent play a game the first time?” After this paper, you can also ask “how quickly does it learn from its mistakes, and does the learned skill generalize?”

Core Mechanism

The heart of the method is the Improvement Dynamics Curve (IDC) harness. Here is the architecture:

        +-----------+   trajectory (screens, actions, rewards)   +-------------+
        | VLM agent | ----------------------------------------> | UE5 Game    |
        +-----------+                                           | Environment |
            ^  <---- actions (unified 2D joystick) ----------- +-------------+
            |                                                       |
            |  failure summary, score, replay                       |
            |                                                       v
            |                                               +---------------+
            +------- refined skill prompt (text) <-------- | Reflector LLM |
                                                           | (with tools:   |
                                                           |  read-failure, |
                                                           |  write-prompt) |
                                                           +---------------+

The agent carries a bounded skill prompt — a short text that encodes its strategy (e.g., “approach enemies from behind, dodge when health < 30%”). This prompt is frozen during a game episode. After each episode, the Reflector LLM (a large language model with tool-access) reads the agent’s failure summary and the environment logs, then writes a new, tighter version of the skill prompt. The prompt is bounded: it cannot grow without limit (max ~500 tokens), forcing the agent to compress what it learns.

The IDC plots score against reflection round. Because each round also runs the agent on held-out task variants (e.g., a different map layout for the same game logic), the curve reveals both learning speed and overfitting.

Structural metaphor: Think of the VLM agent as a student driver and the Reflector LLM as a driving instructor with a giant notebook (tool-use). The student gets a “rule card” (bounded skill prompt) before each drive: “check mirrors every 5 seconds, keep 2-second distance.” After the drive, the instructor reviews the dashcam footage (trajectory), notes failures (near-miss with pedestrian), and writes a revised card for the next drive: “check mirrors every 5 seconds, AND slow down before crosswalks.” The card is bounded — the student can’t memorize a novel-length manual. Over several drives, the IDC shows how the student’s driving score improves, and the held-out test (a rainy road) checks if the skill transfers. The instructor never touches the wheel — it only rewrites the rule card.

Key Concepts

• Bounded Skill Prompt: A compact, editable text (<= 500 tokens) that serves as the agent’s behavioral policy for one game episode. It’s “bounded” because it can’t grow indefinitely — the Reflector must compress insights into a fixed-size memory. Example: for a PvP shooting game, the prompt might start as “move toward enemies, shoot when close” and after reflection become “move toward enemies while staying behind cover, reload after 3 shots.” The bound forces the agent to prioritize its most useful lessons.

• Improvement Dynamics Curve (IDC): A plot of game score (y-axis) vs. reflection round number (x-axis), enabling three observables: (1) initial score, (2) slope (learning rate), (3) asymptotic plateau. The curve exposes whether an agent learns fast but caps low, or learns slowly but keeps improving. Beyond the curve, each round’s held-out score shows if the agent is memorizing or generalizing.

• Held-out Variants: For each of the 12 games, the authors define 2-3 alternative versions (e.g., different enemy density, alternative map geometry, reversed coop roles). After each reflection round, the agent is evaluated on these variants without further prompt updates. This tests whether the skill prompt captures the game’s invariant structure or just exploits the specific training layout.

Framework Shift

Before (mainstream approach):        After (this paper):

Score (y)                            Score (y)
 ^      * (single point)             ^      /---\  (IDC curve)
 |                                    |     /     \
 |                                    |    /       \
 +----> game                          +----> reflection round
                                      |
                                      Also: held-out variant lines
                                      (dashed) to check generalization

One sentence: From a static one-shot score to a dynamic learning curve with generalization probes — the core shift is treating game-playing ability as a learnable property, not a fixed trait.

Expert Assessment

Problem choice: Real gap. The limitations of single-score benchmarks have been discussed in the RL community, but no one had built a practical multi-game, multi-round system for VLMs. The paper correctly identifies that “first-attempt performance” conflates prior training data with reasoning ability, and that learning dynamics matter.

Method maturity: Clever insight in the IDC + bounded prompt idea. The implementation is heavy (UE5 integration, prompt optimization loop with an external LLM), but the concept is elegant. A simpler approach would be to just run multiple trials with random seeds and report distribution — but that doesn’t capture the intentional improvement from reflection. The bounded prompt trick is non-obvious and forces compression, which is neat.

Experimental integrity: Fair baselines? They compare 12 agents (GPT-4V, Gemini, LLaVA, etc.) under the same unified action interface, which is good. However, only 4 top agents are tested under IDC — we need to see IDC for all 12 to understand if low-scoring agents also benefit. The held-out variants seem surface-level (map layout swaps) — would the skills transfer to fundamentally different game logic? No. Red flag: the authors don’t report variance across multiple runs of the same agent under the same condition. Learning is stochastic, a single curve per agent is risky.

Writing quality: The methods section is clear and the ASCII diagrams (in the original paper) are helpful. However, the results section is a laundry list of score tables without deep analysis of *why certain agents learn faster. The section that discusses what patterns the IDC curves reveal (e.g., early plateau for GPT-4V vs. steady improvement for Gemini) is too brief. If they rewrote the discussion to connect curve shapes to agent architecture (e.g., whether the VLM uses tool-calling or not), the paper would be much stronger.

Verdict: weak accept — interesting framework and solid engineering, but insufficient depth in experimental analysis to be a landmark. Worth reading for the IDC concept and the unified benchmark protocol.

Takeaways

• Auto-prompt refinement: Use an LLM as a “self-improvement coach” by having it analyze execution logs and rewrite a bounded policy text. This transfers to any agentic task where the agent’s behavior can be captured in a short prompt (e.g., web navigation, robotic control via natural language).
• IDC as evaluation protocol: For any learned policy, plot score vs. training step *and include held-out variants in the same chart. This divorces the evaluation from a single checkpoint and reveals overfitting.
• Bounded memory trick: Constraining the skill prompt’s length forces the agent to distill its lessons — a simple regularization that can be applied to any learned controller with an editable prompt.