Controllable Traffic Agents with Behavior Latents: Steering Simulation Without Losing Realism

Paper: 2607.02496 Authors: Juanwu Lu, Junyu Zhu, Ziran Wang Categories: cs.RO, cs.LG

The Gap

Here’s the state of play: traffic simulation for autonomous driving has two camps. Camp A builds imitation models—agents that faithfully reproduce logged driving data. These top the Waymo leaderboard for realism but are opaque; you get what you get, no knobs to turn. Camp B builds controllable agents—models you can steer toward specific behaviors (go faster, keep more distance). But these historically sacrifice realism and suffer from “reward hacking,” where agents game the steering signal instead of genuinely changing behavior.

The missing piece: a framework that’s both realistic and interpretable-steerable, with monotone control responses (crank up the speed dial → agent actually goes faster, no weird stalls or gaming). Prior work either picked one side or achieved control through brittle reward thresholds that destroy gradient signal for edge-case agents.

Problem:
  Realistic agents       Controllable agents
  (imitation-based)      (reward-steered)
       |                        |
       v                        v
  Opaque, no knobs       Less realistic,
  to turn                 reward hacking
       |                        |
       +------ gap ------------+
                  |
                  v
  Assumption:
  Per-agent behavior latent can
  encode "driving personality"
  conditioned on channel returns
                  |
                  v
  Method:
  CNeVA - conjugate variational
  inference + rectified flow +
  soft eligibility gates
                  |
                  v
  Evidence:
  Waymo benchmark: competitive
  realism + monotone steering
  on speed, accel, safety
                  |
                  v
  Conclusion:
  Controllability and realism
  need not be a tradeoff

The Increment

One sentence: Before this paper, you had to choose between agents that drive like humans (but can’t be steered) or agents you can steer (but don’t drive like humans); after this paper, you get both—a model that matches top imitation baselines on realism while exposing per-channel control knobs with monotone responses.

Core Mechanism

CNeVA has three interlocking parts. First, a behavior latent ( z ) is inferred for each agent using a conjugate variational update—a closed-form trick that avoids expensive gradient-based inference. The latent is a Gaussian conditioned on per-channel discounted returns (think: accumulated reward for speed, safety, map compliance, etc.). This gives each agent a compact “personality signature.”

Second, a rectified-flow trajectory generator takes the behavior latent, scene context, and channel-specific guidance signals to produce trajectories. The model is trained with a mixed channel-mask curriculum—during training, some channels are randomly masked out so the model learns to generate with or without steering signals. At inference, classifier-free guidance lets you dial specific channels up or down without needing a separate classifier network.

Third, soft eligibility gates replace hard binary thresholds for deciding which agents count toward each channel’s reward. Instead of agent distance < 5m → eligible (1), else → ineligible (0), CNeVA uses a smooth exponential decay. This preserves gradient signal for agents near the threshold, preventing the dead-zone problem that causes reward hacking in prior work.

[Scene Context]----+
                    |
                    v
[Agent i]--->[Conjugate Variational Inference]---> z_i ~ N(mu, sigma)
                    |                                   |
                    |   (conditioned on per-channel     |
                    |    discounted returns R_c)         |
                    v                                   v
[Channel Mask]--->[Rectified Flow Generator]<---[Behavior Latent z_i]
                       |                               |
                       v                               v
                  [Trajectory]              [Classifier-Free Guidance]
                       |                          |
                       +----------+---------------+
                                  |
                                  v
                         [Steered Output]
                                  |
                                  v
                    [Soft Eligibility Gate]---> [Channel Reward R_c]
                         (exp decay,            (feeds back to
                          not hard threshold)     inference)

Structural metaphor: A professional mixing board.

Imagine a recording studio. Each instrument (agent) comes in on its own channel. The mixing board (CNeVA) has faders for each aspect of the sound: bass (speed), treble (aggression), reverb (safety margin). The behavior latent is like the “preset” for each instrument—a compressed signature of its character.

The conjugate variational inference is the efficient sound engineer who, instead of painstakingly adjusting each knob through trial and error, calculates the optimal settings directly from the instrument’s frequency profile (the discounted returns). Closed-form, no iterative fiddling.

The rectified flow generator is the signal path: it takes the preset and the fader positions and produces the final mix (trajectory). The mixed channel-mask curriculum is like training the engineer with some faders taped over—forcing them to produce good mixes even when some controls are unavailable. At inference, classifier-free guidance is like having a “solo” button: you boost one channel’s influence without a separate “is this channel important?” detector.

The soft eligibility gates are the smooth crossfaders. In a crude setup, instruments are either “in the mix” or “muted”—a hard cutoff that creates jarring transitions. CNeVA’s crossfaders blend smoothly, so an instrument at the threshold of inclusion still contributes a gentle signal, preventing the mix from suddenly going silent or jumping to full volume (reward hacking).

Without this metaphor, you’re staring at equations. With it, you can retell the whole system: “It’s a mixing board where each agent gets a preset, the engineer computes settings in closed form, the signal path is a rectified flow, and the crossfaders are smooth to prevent audio glitches.”

Key Concepts

• Behavior Latent: Think of each driver as having an invisible “driving personality dial” set somewhere between cautious and aggressive. In CNeVA, this dial position is a point in a smooth probability cloud (Gaussian). The model learns to read each agent’s past rewards—how fast they’ve been going, how close they’ve been to collisions, how well they’ve stayed in lanes—and infers where their dial is set. Crucially, the dial is *per-agent, so two cars in the same scene can have different personalities. Concrete example: Agent A has been braking hard near pedestrians → its latent shifts toward “cautious.” Agent B has been cruising fast in open road → its latent shifts toward “assertive.” The model then generates futures consistent with each personality.

• Soft Eligibility Gates: Imagine grading student exams with a rule: “If score ≥ 70, count the student; otherwise, ignore them.” Students at 69 get zero credit for improvement—they’re invisible to the grading system. This is a hard eligibility threshold, and it’s what prior controllable sim agents used. CNeVA replaces it with a smooth curve: a student at 69 gets 99% credit, at 65 gets 75%, at 50 gets 10%. The gradient signal is preserved everywhere, so the optimizer can still nudge borderline students upward. In driving: an agent 5.1m from a collision threshold (barely “safe”) gets almost-full safety reward, not zero. This prevents the model from learning to game the threshold boundary.

• Classifier-Free Guidance (CFG): Normally, if you want to steer a generative model with a condition (e.g., “make this agent go fast”), you need a classifier that scores how well the output matches the condition. CFG eliminates the classifier. Instead, during training, you randomly drop the conditioning signal (mask out the “go fast” channel) so the model learns both conditioned and unconditioned generation. At inference, you extrapolate *away from unconditioned toward conditioned by amplifying the difference: output = unconditioned + w × (conditioned - unconditioned), where w > 1 is your steering strength. It’s like seasoning: the model learned to cook with and without salt; at serving time, you add extra salt to taste.

Framework Shift

Before (mainstream):                 After (CNeVA):

[Imitation Model]                    [CNeVA]
     |                                   |
     v                                   v
Learn from demos                   Learn from demos + returns
     |                                   |
     v                                   v
Generate realistic                 Infer per-agent behavior
trajectories                       latent (closed-form)
     |                                   |
     v                                   v
[END - no steering]                [Rectified Flow Generator]
                                        |
                                        v
                                   Classifier-free guidance
                                   with channel masks
                                        |
                                        v
                                   Soft eligibility gates
                                   (smooth reward signal)
                                        |
                                        v
                                   Steerable + Realistic

From opaque imitation to interpretable steering, the core shift is encoding “driving personality” as an explicit latent variable inferred from returns, then using smooth guidance and eligibility to make the steering monotone and reward-hack-resistant.

Expert Assessment

Problem choice: This is a real gap. The field has been stuck on the realism-vs-controllability tradeoff for years—Waymo’s leaderboard rewards realism but doesn’t test steerability, and controllable models don’t compete on realism benchmarks. The paper sits at a natural junction in the field’s trajectory where simulation-based AV testing is maturing and needs both properties. Not manufactured; genuinely useful for practitioners who run regression tests on sim agents.

Method maturity: The conjugate variational update is clever—using the Gaussian-conjugate structure to get a closed-form posterior avoids the instability of amortized inference for this particular structure. The soft eligibility gate is a simple but effective fix; honestly, it’s surprising prior work didn’t try this. The mixed channel-mask curriculum for CFG is borrowed from diffusion/flow-model tricks (cf. Ho & Salimans 2022) but applied thoughtfully. Not brute force; the pieces fit together well. One concern: the conjugate update assumes the return structure is well-behaved enough for a Gaussian posterior—this may break in more complex reward landscapes.

Experimental integrity: Baselines are reasonable: CTG++ (controllable), MTR/SceneDiffuser (imitation). The paper reports realism metrics where they match top imitation models, and controllability metrics where they expose monotone responses. The key claim—competitive realism—holds up; they’re not #1 on the leaderboard but within striking distance of models that have zero controllability. The monotone steering results are clean. One red flag: the “context-residual return measure” for map compliance is non-standard and hard to compare to prior work. The authors acknowledge steering metrics need physical-plausibility guardrails, which is honest but also means some of their controllability gains may be partially illusory without those guardrails.

Writing quality: The paper reads well for a methods paper, but Section 4 (experiments) could be tighter. The ablation on soft eligibility is buried; it deserves more prominence since it’s the key technical contribution that separates this from “apply CFG to traffic sim.” The reward-hacking analysis is insightful but scattered across sections—pulling it into a dedicated subsection would strengthen the narrative.

Verdict: weak accept — The combination of closed-form inference, soft eligibility, and CFG for multi-channel steering is genuinely novel and practically useful, but the non-standard evaluation metrics and the question of whether the approach scales beyond Waymo’s relatively structured scenarios keep it from a strong accept.

Takeaways

Three concrete things to steal:

1. Soft eligibility gates for any thresholded reward: If your RL or imitation-from-feedback setup has hard eligibility thresholds (e.g., “only count trajectories within X distance”), replace them with exponential decay. This is a one-line change that preserves gradient signal for borderline cases and prevents threshold-gaming. Applies anywhere: robotics grasping success thresholds, game AI score cutoffs, medical decision rules.

2. Conjugate variational inference when your likelihood is Gaussian: If your latent variable model has a Gaussian observation model, don’t amortize the posterior—use the closed-form conjugate update. It’s more stable and cheaper. This is a reminder to check your model’s conjugacy structure before defaulting to an encoder network.

3. Channel-mask curriculum for classifier-free guidance in non-diffusion settings: CFG was born in diffusion models, but the random-channel-masking training trick transfers to any conditional generator (rectified flows, autoregressive models). If you need multi-dimensional steering without a classifier, this curriculum approach is the key enabler.