Self-Conditioned GANs Learn Trajectory Patterns Without Labels or Maps

Paper: 2603.08658 Authors: Tiago Rodrigues de Almeida, Eduardo Gutierrez Maestro, Oscar Martinez Mozos Categories: cs.LG

The Gap

Trajectory forecasting methods typically fall into two camps: supervised approaches that require labeled behavioral modes (walking, turning, stopping), or context-aware methods that need scene information like maps and semantic layouts. Both have problems. Supervised labels are expensive and often miss rare behaviors. Context-dependent methods fail when you don’t have detailed scene data—think surveillance footage from a single camera or GPS traces without map access.

The specific limitation: existing context-free methods struggle with multimodal predictions (multiple plausible futures) and perform poorly on underrepresented behavioral patterns. They either collapse to average predictions or need explicit mode labels.

Problem: Predict future trajectory from past positions
   |
   v
Limitation: Context-free + Unsupervised + Multimodal = Hard
   |
   +---> Prior work: Supervised modes (expensive labels)
   +---> Prior work: Context-aware (needs maps/scenes)
   |
   v
Assumption: Behavioral modes exist in discriminator feature space
   |
   v
Method: Self-conditioned GAN discovers modes without labels
   |
   v
Evidence: Outperforms on rare behaviors (human + road agents)
   |
   v
Conclusion: Self-conditioning enables context-free mode discovery

The Increment

One sentence: Before this paper, context-free trajectory forecasting either needed labeled behavioral modes or collapsed to unimodal predictions; after, a self-conditioned GAN can discover diverse behavioral modes unsupervised and excel at rare patterns.

Core Mechanism

The method uses a GAN where both generator and discriminator are conditioned on a mode variable, but here’s the twist: the mode isn’t provided by labels—it’s learned by the network itself. The generator takes past trajectory positions and a mode code, then produces future positions. The discriminator evaluates whether trajectories are real or fake AND implicitly learns to associate mode codes with behavioral patterns in its feature space.

Three training settings explore this self-conditioning: (1) standard self-conditioned GAN where modes emerge naturally, (2) auxiliary classifier variant where the discriminator explicitly predicts mode labels, and (3) a hybrid approach. The key is that the conditioning creates a feedback loop—the generator learns to produce distinct behaviors for different mode codes, while the discriminator learns to recognize these distinctions, all without external supervision.

The architecture processes 2D coordinate sequences. Past positions go through an encoder, get concatenated with a mode vector, then a decoder generates future positions. The discriminator sees the full trajectory (past + future) and outputs real/fake scores. During training, the mode variable acts as a knob that the generator learns to turn to produce different behavioral patterns.

Training Loop:
                                    
  Past Trajectory ----+             
                      |             
  Mode Code (z) ------+---> Generator ---> Future Trajectory
                      |                           |
                      |                           v
                      +---------------> Discriminator
                                            |
                                            v
                                    Real/Fake + Mode Info
                                            |
                                            v
                                    Backprop Updates Both

Think of it like a jazz musician learning to improvise. You don’t give them a rulebook of “blues style” vs “bebop style”—instead, you play them examples and let them discover that certain note patterns cluster together. The self-conditioning is like the musician developing an internal sense of “if I’m in mode A, these phrases work; if I’m in mode B, those phrases work.” The generator is the musician, the discriminator is their ear, and the mode variable is their intuition about which style they’re playing. Over time, distinct styles emerge from the data itself, not from labeled instruction.

Key Concepts

• Self-conditioning: Imagine training a dog to fetch different objects, but instead of you labeling “ball” vs “stick,” the dog develops its own internal categories. You just reward successful fetches. Self-conditioning means the model conditions its behavior on variables it learns to define itself. The mode code starts as random noise, but through training, the generator learns “mode 3 means sharp turns” and “mode 7 means straight paths” without anyone telling it those labels. The conditioning creates structure where none existed.

• Context-free forecasting: You’re predicting where someone will walk next, but you can’t see if there’s a wall ahead or a door to the right—just their past positions as (x,y) coordinates. Context-free means no semantic scene information, no maps, no object locations. It’s pure geometry. This matters because most real-world data lacks rich context: a single camera view, GPS traces, sensor logs. The challenge is that without context, you must infer behavioral intent purely from motion patterns.

• Mode collapse vs mode discovery: Standard GANs often suffer mode collapse—the generator learns to produce only average, safe outputs because that fools the discriminator. If you’re predicting pedestrian paths, it might always predict “walk straight” because that’s common. Mode discovery is the opposite: finding and preserving diverse behavioral patterns. The self-conditioning acts as a forcing function—by tying generation to mode codes, the model must learn distinct behaviors for each code, preventing collapse to the mean.

Framework Shift

Before (mainstream approach):        After (this paper):

Labels --> Model --> Prediction      Trajectories --> Self-Conditioned
  or                                                  GAN
Scene Context --> Model                                |
                                                       v
                                              Discovered Modes
                                                       |
                                                       v
                                                  Prediction

Requires: External information      Requires: Only trajectory data
Learns: Mapping from labels/context  Learns: Modes + Prediction jointly
Fails: When labels/context missing   Fails: When patterns too complex

[One sentence: From externally-defined behavioral categories to self-discovered motion patterns, the core shift is learning what the modes are simultaneously with learning to predict.]

Expert Assessment

Problem choice: Real gap. Context-free unsupervised trajectory forecasting is genuinely useful—think privacy-preserving systems, resource-constrained deployments, or domains where labeling is prohibitively expensive. The focus on rare behavioral patterns is smart; that’s where supervised methods with imbalanced data struggle most. This sits in the practical middle ground between pure research and deployment needs.

Method maturity: Self-conditioned GANs aren’t new (InfoGAN, AC-GAN exist), so this is more application engineering than fundamental innovation. The three training settings feel like hyperparameter exploration rather than principled design. A simpler approach might be clustering in latent space post-hoc, though the end-to-end learning has appeal. The method is clever but not groundbreaking.

Experimental integrity: Two datasets (human motion, road agents) is reasonable but not extensive. The claim “outperforms on least representative labels” is interesting but needs scrutiny—are they cherry-picking metrics? The abstract lacks quantitative results, which is a yellow flag. Fair baselines would compare against other unsupervised context-free methods, but the paper doesn’t specify what those are. The “performs well globally” language is vague.

Writing quality: The abstract is too high-level—what are the three training settings? What’s the actual performance gain? The phrase “context-free unsupervised approach” is good, but “self-conditioned GAN” assumes reader familiarity. The results section likely needs tightening; academic papers often bury key findings in tables. If I were reviewing, I’d ask for clearer problem motivation and more detailed baseline comparisons.

Verdict: weak accept — Addresses a real problem with a reasonable (if incremental) approach, but lacks the experimental depth and novelty for strong acceptance.

Takeaways

The self-conditioning trick is portable: any generative model that struggles with mode collapse could benefit from conditioning on learned latent codes rather than external labels. If you’re working with imbalanced datasets where rare classes matter (fraud detection, medical diagnosis, anomaly detection), this “discover modes unsupervised” approach could help.

The context-free framing is valuable beyond trajectories. Anytime you have sequential data without rich metadata—sensor streams, user behavior logs, financial transactions—you can apply this pattern: learn behavioral modes from the sequences themselves rather than requiring external categorization.

Practically, the three training settings suggest that self-conditioning isn’t one-size-fits-all. You’ll need to experiment with how tightly to couple mode discovery with the main task. That’s annoying but honest—there’s no free lunch.