CaMBRAIN: Streaming EEG Analysis with Causal State Space Models

Paper: 2605.28792 Authors: Abhilash Durgam, Nyle Siddiqui, Jeffrey A. Chan-Santiago, Qiushi Fu, Elakkat D. Gireesh, Mubarak Shah Categories: cs.AI, cs.HC, cs.LG

The Gap

Existing EEG analysis models hit a wall with long recordings. Attention-based architectures (the current standard) scale quadratically with sequence length — a 10-minute EEG at 256 Hz means 153,600 time steps, making full-sequence processing prohibitively expensive. The workaround? Sliding windows. But this fragments the signal into isolated chunks, preventing the model from seeing patterns that span minutes (like seizure buildup or sleep stage transitions).

Bidirectional models like BIOT and LaBraM process signals in both directions, which seems sensible for offline analysis but is fundamentally incompatible with real-time streaming. You can’t look into the future when the future hasn’t happened yet. The field lacks a model that can (1) process arbitrarily long EEG in linear time, (2) maintain global context across the entire recording, and (3) operate causally for real-time deployment.

Problem: Long EEG (hours) + Attention (O(n^2)) = Infeasible
         |
         v
Assumption: Causality is sufficient (no future needed)
         |
         v
Method: Causal SSM (Mamba) + Multi-stage self-supervised training
         |
         v
Evidence: SOTA on 3 datasets, 10x throughput, linear scaling
         |
         v
Conclusion: First streaming-capable EEG foundation model

The Increment

One sentence: Before this paper, EEG models chose between global context (too slow) or real-time processing (no long-range memory); after, a single causal model does both with linear complexity.

Core Mechanism

CaMBRAIN replaces attention with a Mamba-based state space model (SSM) that processes EEG causally — left to right, never looking ahead. The core is a selective state space: at each time step, the model updates a hidden state vector based on the current input and previous state. Unlike RNNs, SSMs use structured matrices that enable parallel training while maintaining the recurrent inference property needed for streaming.

The architecture stacks Mamba blocks with residual connections. Each block contains: (1) a selective SSM layer that updates the hidden state, (2) a gating mechanism that decides what information to retain or discard, and (3) layer normalization. Input EEG (raw voltage time series) passes through a patch embedding layer that converts short segments into tokens, then flows through the Mamba stack, producing representations at each time step.

Raw EEG: [v1, v2, v3, ..., vn]
         |
         v
Patch Embed: [e1, e2, ..., em]  (m < n, non-overlapping windows)
         |
         v
    +---[Mamba Block 1]---+
    |   SSM + Gate + Norm |
    +----------+-----------+
               |
    +---[Mamba Block 2]---+
    |   SSM + Gate + Norm |
    +----------+-----------+
               |
              ...
               |
    +---[Mamba Block L]---+
    |   SSM + Gate + Norm |
    +----------+-----------+
               |
               v
    Hidden States: [h1, h2, ..., hm]
               |
               v
    Task Head (classification/regression)

Think of CaMBRAIN as a conveyor belt with memory stations. Raw EEG is the stream of items moving down the belt. Each Mamba block is a worker station that examines the current item, checks a notepad (hidden state) containing summaries of all previous items, updates the notepad with new information, and passes both the item and updated notepad to the next station. The notepad is compact — it doesn’t store every detail, only what the worker learned to remember through training. Crucially, workers never peek ahead at items still upstream; they only see what’s already passed through. The final station reads the notepad and makes a decision (classify the EEG segment). Because each station operates in constant time and stations work in parallel during training, the whole belt scales linearly with the number of items.

Key Concepts

• State Space Models (SSMs): Imagine you’re tracking a ball’s position over time, but you can only measure it noisily. An SSM maintains a “hidden state” (your best guess of the true position and velocity) that gets updated each time you receive a new measurement. The update rule is a linear transformation: new_state = A × old_state + B × input. For EEG, the “hidden state” is a learned representation of everything important that happened so far, and the update rule is learned from data. The key advantage: you can compute all updates in parallel during training (using convolution tricks), but at inference time, you only need the previous state — perfect for streaming.

• Causal vs Bidirectional Processing: Causal means “only use the past.” When processing time step t, a causal model only sees inputs from steps 1 to t. Bidirectional models see the entire sequence (past and future) before making any decision. For offline analysis, bidirectional seems better — more information, better accuracy. But for real-time systems (seizure detection, brain-computer interfaces), you can’t wait for the future. The paper’s insight: EEG is inherently causal (brain activity unfolds forward in time), so bidirectional processing is overkill. A well-trained causal model can match or exceed bidirectional performance while enabling streaming.

• Multi-stage Self-Supervised Training: Training a streaming SSM on EEG is tricky. Standard self-supervised objectives (mask and reconstruct) don’t explicitly teach the hidden state to remember long-range context — they optimize for local reconstruction. CaMBRAIN uses three stages: (1) Masked reconstruction on short segments to learn local patterns, (2) Contrastive learning across long windows to force the hidden state to encode information that distinguishes distant time points, (3) Fine-tuning on downstream tasks. Stage 2 is critical: by contrasting representations separated by minutes, the model learns to retain salient events in its hidden state, not just recent inputs.

Framework Shift

Before (Attention-based):              After (CaMBRAIN):

Full EEG sequence                      Full EEG sequence
[===================]                  [===================]
         |                                      |
    Chunk into                            Process causally
    overlapping                           left-to-right
    windows                                    |
         |                                     v
    +----+----+----+                      h1->h2->h3->...->hn
    |    |    |    |                      (hidden state flows)
    v    v    v    v                           |
  [Attn][Attn][Attn]...                       v
    |    |    |    |                      Single global
    v    v    v    v                      representation
  Isolated predictions                         |
  (no global context)                          v
                                          Streaming inference
  O(n^2) per window                       O(n) total
  Cannot stream                           Real-time capable

One sentence: From fragmenting long EEG into isolated attention windows to maintaining a single causal hidden state that flows through the entire recording, the core shift is global memory without quadratic cost.

Expert Assessment

Problem choice: Real gap. Clinical EEG recordings routinely span hours (overnight sleep studies, long-term epilepsy monitoring), and existing models genuinely struggle with this. The attention bottleneck is well-documented, and sliding windows do lose global context. The problem sits at the intersection of clinical need (real-time monitoring) and architectural limitation (quadratic scaling). Not manufactured.

Method maturity: Solid engineering, not groundbreaking science. Mamba (the SSM backbone) already existed; the contribution is adapting it to EEG and designing a training pipeline that makes it work. The multi-stage training is clever — contrastive learning across long windows is a principled way to encourage long-range memory. But there’s no deep theoretical insight here. The paper doesn’t explain *why SSMs should outperform attention on EEG beyond computational efficiency. Could a well-designed linear attention variant achieve similar results? Unclear.

Experimental integrity: Baselines are fair (BIOT, LaBraM, BENDR — current SOTA). Three datasets (TUH Abnormal, SHHS, Sleep-EDF) cover different tasks (abnormality detection, sleep staging). Results are convincing: SOTA accuracy with 10x throughput. However, the paper doesn’t ablate the multi-stage training — how much does each stage contribute? Also, all experiments use relatively short evaluation windows (30-60 seconds). The claim of “hours-long” capability is supported by throughput numbers, not end-to-end experiments on truly long recordings. Minor red flag.

Writing quality: Clear motivation and method description. The multi-stage training pipeline is well-explained. Weakness: the related work section is thin — doesn’t engage deeply with prior SSM work or explain why previous SSM attempts on time series failed. The results section focuses heavily on throughput but underexplores failure modes. Which types of EEG events does the causal model miss compared to bidirectional baselines? Rewriting Section 4 (Results) to include error analysis would elevate the paper significantly.

Verdict: Weak accept — Solves a real problem with solid engineering, achieves strong empirical results, and enables a new capability (streaming EEG inference). But the method is incremental (apply existing SSM to new domain), and the evaluation could be more thorough. Useful contribution, not a landmark.

Takeaways

For practitioners:

1. Causal is enough for time series: If your data has a natural temporal direction (sensor streams, logs, medical signals), don’t default to bidirectional models. A well-trained causal model can match accuracy while enabling streaming deployment.
2. Self-supervised objectives matter for SSMs: Standard masked reconstruction doesn’t teach SSMs to remember long-range context. Add a contrastive stage that explicitly forces the hidden state to encode information across long intervals.
3. Linear complexity unlocks new use cases: The 10x throughput isn’t just faster — it makes previously infeasible applications viable (real-time monitoring of hours-long recordings, edge deployment on low-power devices).

Transferable technique: The multi-stage training pipeline (local reconstruction → long-range contrastive → task fine-tuning) is domain-agnostic. Apply it to any streaming time series where rare, brief events are separated by long intervals (network intrusion detection, industrial sensor monitoring, financial tick data).