MOJO: Teaching Neural Decoders to Learn Without Labels

Paper: 2607.14086 Authors: Ximeng Mao, Nanda H. Krishna, Avery Hee-Woon Ryoo, Matthew G. Perich, Guillaume Lajoie Categories: cs.LG, q-bio.NC

The Gap

Recent work introduced a powerful idea: tokenize neural spike trains the way NLP tokenizes words, then train transformer-style models to decode behavior. This “spike tokenization” approach enables multi-session pretraining and achieves state-of-the-art decoding accuracy. But there’s a catch — every one of these models is trained purely with supervised learning (SL). That means you need behavioral labels (what was the animal doing at each moment?) paired with every snippet of neural data you train on. Labels are expensive: they require carefully designed experimental paradigms, human annotation, or synchronized behavioral recordings. Enormous datasets of neural recordings exist without such labels, and they sit unused.

The logical path from this limitation to MOJO’s contribution:

Spike-tokenizing models exist
        |
        v
But they only use supervised learning (SL)
        |
        v
SL requires paired behavioral labels for every training sample
        |
        v
Massive unlabelled neural datasets are wasted
        |
        v
MOJO: add masked autoencoder (SSL) alongside SL
        |
        v
Joint training uses both labelled AND unlabelled data
        |
        v
Better performance, especially when labels are scarce
        |
        v
+ interpretable representations, + cross-modal generalization

The Increment

One sentence: Before this paper, spike-tokenizing neural decoders could only learn from labelled data; after this paper, they can also learn from unlabelled data via masked autoencoding, dramatically improving performance when labels are scarce.

Core Mechanism

MOJO’s architecture has two training objectives applied jointly to the same spike-tokenizing backbone. The backbone — a transformer that ingests tokenized spike events from multi-electrode recordings — is shared between both objectives. During training, each batch of neural data flows through two parallel paths:

Path 1 — Supervised Learning (SL): Spike tokens from a labelled session are fed through the transformer to produce a hidden representation, which is projected onto a behavioral output (e.g., hand velocity during reaching). Standard regression loss is computed.

Path 2 — Self-Supervised Learning (SSL): A portion of spike tokens are randomly masked (analogous to BERT’s masked language modeling or MAE in vision). The same transformer receives the corrupted sequence and must reconstruct the masked tokens from context. No labels are needed — the “answer” is the original spike data itself.

The total loss is a weighted combination of the SL and SSL objectives. During pretraining, you can throw in large amounts of unlabelled data (only SSL loss applies to those samples) alongside smaller labelled datasets (both losses apply). During fine-tuning on a new session with limited labels, the pretrained model transfers better because it has already learned meaningful neural representations through SSL.

                  +------------------+
                  |  Neural Spikes   |
                  |  (tokenized)     |
                  +--------+---------+
                           |
                  +--------v---------+
                  |   Random Mask    |
                  |   a portion of   |
                  |   spike tokens   |
                  +--------+---------+
                           |
              +------------+------------+
              |                         |
     +--------v--------+     +---------v---------+
     |  Masked Tokens   |     |  Unmasked Tokens  |
     |  (corrupted seq) |     |  (full seq)        |
     +--------+---------+     +---------+----------+
              |                         |
     +--------v--------+     +---------v----------+
     |                 |     |                    |
     |   Backbone      |     |   Backbone         |
     |   Transformer   |     |   Transformer      |
     |   (shared)      |     |   (shared)         |
     +--------+--------+     +---------+----------+
              |                         |
     +--------v--------+     +---------v----------+
     |  Reconstruct    |     |  Decode behavior   |
     |  masked tokens  |     |  (e.g. velocity)   |
     +--------+--------+     +---------+----------+
              |                         |
     +--------v--------+     +---------v----------+
     |  SSL Loss       |     |  SL Loss           |
     |  (no labels     |     |  (needs labels)    |
     |   needed!)      |     |                    |
     +--------+--------+     +---------+----------+
              |                         |
              +------------+------------+
                           |
                  +--------v---------+
                  |  Combined Loss   |
                  |  = w1*SSL + w2*SL|
                  +------------------+

Structural metaphor: Think of a medical residency. A young doctor (the transformer) can learn medicine in two ways. The supervised path is like sitting the board exam: you’re given a patient case and must produce the correct diagnosis — you learn, but only from cases your attending physician has graded. The self-supervised path is like clinical rotations where you see hundreds of patients daily, take histories, observe symptoms, and try to predict what’s wrong before the lab results come back. Nobody gives you the answer immediately, but the act of prediction from context teaches you deep patterns of disease. MOJO does both simultaneously: the model studies for exams (SL on labelled data) while also doing clinical rounds (SSL on unlabelled data). The result is a doctor who performs better on exams — especially when there are few practice cases available — and who develops an intuitive understanding of the underlying biology that shows up in unexpected ways (like better organ-system classification without ever being taught it explicitly).

Key Concepts

• Spike Tokenization: Traditional neural decoding treats each time bin as a vector of firing rates across electrodes. Spike tokenization instead converts each individual spike event into a discrete token — like converting a speech waveform into a sequence of phonemes. This is powerful because it preserves temporal precision and allows transformer architectures (designed for sequences) to be applied directly. Imagine recording from 64 electrodes over 2 seconds: instead of getting a 64×100 rate matrix (100 time bins), you get a variable-length sequence of “who spiked when” tokens that the transformer can attend over.

• Masked Autoencoder (MAE) for Neural Data: The core SSL trick. You take a complete spike sequence, randomly hide ~40-80% of the tokens, and train the model to fill in the blanks. The model must learn the statistical structure of neural population activity — which neurons tend to co-fire, what temporal patterns precede specific behaviors, how activity relates to brain region — just from the unmasked context. It’s like solving a crossword puzzle: the constraints of the surrounding words force you to learn deep patterns of language, even though nobody explicitly taught you grammar rules.

• Few-Shot Fine-Tuning: After pretraining MOJO on a large dataset, you encounter a new experimental session with a different animal, different electrode placement, or different task — and you only have 5 minutes of labelled data. You fine-tune the pretrained model on this tiny labelled set. The SSL pretraining pays off enormously here because the model has already learned general features of neural activity; it just needs a small nudge to adapt to the specifics of the new session. Without SSL, the model must learn neural representations and behavioral mapping simultaneously from scratch with very few examples — a much harder problem.

Framework Shift

Before (mainstream approach):            After (MOJO):

  Labelled neural data only              Labelled + Unlabelled data
         |                                      |
         v                                      v
  +----------------+                    +----------------+
  |  SL objective  |                    | SL + SSL       |
  |  (regression   |                    | objectives     |
  |   to behavior) |                    | (joint train)  |
  +----------------+                    +----------------+
         |                                      |
         v                                      v
  Decoder trained                        Decoder trained
  on labelled data                       on ALL data
  only                                   (labelled + not)
         |                                      |
         v                                      v
  Limited to datasets                    Leverages unlabelled
  with behavioural                       neural recordings
  labels                                 for richer features

From “label-gated training” to “open-data training,” the core shift is that self-supervised pretraining on spike sequences lets the model extract useful neural features without any labels, and joint optimization with SL means labelled data still guides task-relevant learning — you get the best of both worlds in a single framework.

Expert Assessment

Problem choice: This is a real and well-motivated gap. The spike tokenization literature (including the authors’ own prior work) has been explicitly SL-only, and the NLP/vision communities demonstrated years ago that SSL dramatically improves data efficiency. Applying this to neural decoding is a natural and overdue step. The gap sits squarely in the mainstream trajectory of neural foundation models — this is the obvious next move, and someone had to make it rigorously.

Method maturity: The approach is conceptually straightforward — it’s a masked autoencoder added to an existing supervised pipeline. This is a “clever incremental” rather than a “radical new paradigm.” The insight is less about architectural novelty and more about recognizing that the existing MAE framework transfers cleanly to spike tokens. There’s a risk that simpler alternatives (e.g., contrastive learning, or even just pretraining with next-spike prediction) might work comparably well, but the authors don’t explore these. Still, masked autoencoding is well-understood and reliable, so the choice is defensible.

Experimental integrity: The experiments are solid in breadth — three datasets across species (monkey, mouse, human) and modalities (spikes, ECoG). The few-shot fine-tuning evaluation is the most compelling result and directly addresses the claimed benefit. The comparisons against purely SL-trained baselines are fair (same architecture, same data, minus SSL). The inclusion of ECoG generalization is a nice touch that demonstrates the method isn’t spike-specific. One concern: the paper claims performance “comparable to neuro-foundation models designed specifically for continuous signals” for ECoG, but this comparison deserves more scrutiny — are those NFMs using comparable amounts of training data? The paper could be more transparent about this.

Writing quality: The paper reads clearly and follows a logical structure. The weakest section is the related work, which could better position MOJO against the broader self-supervised learning literature in neuroscience (e.g., contrastive methods on calcium imaging). The ablations on SSL weight and masking ratio are present but could be more thorough — how sensitive is MOJO to these hyperparameters across different datasets?

Verdict: weak accept — The contribution is solid incremental work that answers an obvious question with clean experiments across multiple datasets, but lacks the architectural innovation or surprising insight that would make it a standout. It’s the kind of paper the field needed to have written.

Takeaways

For neural engineers building BCIs: The most concrete takeaway is the few-shot fine-tuning recipe. If you’re deploying a BCI decoder that needs to adapt to a new user or a new day’s recording, pretrain with MOJO’s joint objective on your existing large dataset, then fine-tune with just a few minutes of calibration data. The SSL component meaningfully reduces the labelled data requirement.

For ML practitioners in other domains: The principle — “add a masked reconstruction objective to any supervised sequence model to exploit unlabelled data” — is domain-agnostic. If you’re working with any sequential data where labels are expensive but raw data is plentiful (e.g., sensor streams, financial time series, seismology), this joint-training recipe transfers directly.

For computational neuroscientists: The finding that SSL-trained representations improve brain region classification and spike-statistics prediction without explicit supervision suggests these models are learning genuine neurobiological structure, not just task-relevant shortcuts. This has implications for using trained models as scientific tools, not just engineering artifacts.