Paper: 2605.13833 Authors: Hoang-Quan Nguyen, Sankalp Pandey, Khoa Luu Categories: cs.LG, cs.CV
The Gap
Transformers model long sequences through attention but hit a wall: quadratic cost means 10,000 tokens becomes computationally prohibitive. State-space models (SSMs) like S4 and Mamba solve the cost problem with linear-time recurrence, but their memory updates are additive or linear—each new token just gets added to a running sum. This limits their ability to represent complex, non-linear interactions across distant tokens.
The gap: we have efficient memory (SSMs) or expressive memory (attention), but not both. SSMs update state through h_t = A·h_{t-1} + B·x_t, a fundamentally linear operation. Can we enrich the state representation without sacrificing linear complexity?
Problem: Long sequences need global context
|
v
Constraint: Quadratic attention vs linear but limited SSMs
|
+---> Transformers: O(n^2) attention
| |
| v
| Captures global interactions but doesn't scale
|
+---> SSMs: O(n) recurrence
|
v
Scales but memory = additive updates
|
v
Assumption: Quantum superposition can encode richer state
|
v
Method: QLAM - quantum state as memory, parameterized circuits as updates
|
v
Evidence: Beats SSMs and transformers on sequential image tasks
|
v
Conclusion: Quantum memory > classical linear memory for sequence modeling
The Increment
One sentence: Before QLAM, sequence models chose between quadratic expressiveness (attention) or linear simplicity (SSMs); after QLAM, quantum superposition offers a third path—linear time with non-classical memory capacity.
Core Mechanism
QLAM replaces the classical hidden state in SSMs with a quantum state. Instead of a vector of real numbers, the memory is a quantum state |ψ⟩ whose amplitudes encode a superposition of historical information. At each timestep, a parameterized quantum circuit (PQC) acts on this state, conditioned on the input token. This is the update step—analogous to h_t = f(h_{t-1}, x_t) in classical recurrence, but f is now a quantum operation that can create entanglement and interference.
To extract information, QLAM uses query-dependent measurements. Given a query (e.g., from the current token), the system measures the quantum state in a basis determined by that query. The measurement collapses the superposition and yields classical outputs that feed into downstream layers. This measurement step is where global dependencies surface: because the quantum state evolved through all previous tokens, the measurement can reveal correlations that additive updates would miss.
The architecture has three stages: (1) encode input token into quantum parameters, (2) apply PQC to evolve quantum state, (3) measure state with query-dependent observables to produce output. Crucially, steps 1-3 happen in sequence, so total cost is O(n) for n tokens—same as SSMs, but the quantum state can represent exponentially more configurations than a classical vector of the same dimension.
Input token x_t
|
v
[Encoder] --> theta_t (quantum parameters)
|
v
|psi_{t-1}> --[PQC(theta_t)]--> |psi_t>
^ |
| v
+--- (quantum state memory) ---+
|
v
Query q_t --> [Measurement basis]
|
v
Observable <O_q>
|
v
Classical output y_t
Think of QLAM as a holographic film. In classical SSMs, each frame of a movie is stored by adding ink to a canvas—new information overwrites or blends with old. In QLAM, each frame is recorded as an interference pattern on holographic film. When you shine a laser (the query) through the film, the interference pattern reconstructs a 3D image—not by reading pixels sequentially, but by letting all the overlapping waves interact. The quantum state is the hologram: it holds all past tokens in superposition. The measurement is the laser: it extracts task-relevant information by probing the interference pattern. The PQC is the recording process: it encodes each new token by modulating the phase and amplitude of the quantum state, creating constructive and destructive interference that encodes relationships between tokens.
Key Concepts
-
Quantum superposition: A classical bit is 0 or 1. A quantum bit (qubit) can be in a state α|0⟩ + β|1⟩, where α and β are complex numbers satisfying |α|² + |β|² = 1. This means the qubit is “both 0 and 1” until measured. For n qubits, the state space has 2^n dimensions—exponentially larger than n classical bits. In QLAM, this exponential capacity lets a small quantum state encode rich, entangled representations of sequence history. Concretely: 10 qubits can represent 1024 basis states simultaneously, whereas 10 classical bits can only be in one of 1024 states at a time.
-
Parameterized quantum circuit (PQC): A sequence of quantum gates (rotations, entangling operations) whose angles are learned parameters. Think of it as a neural network layer, but operating on quantum states. In QLAM, the PQC takes the current quantum state and input token, applies rotations conditioned on the token, and outputs an updated quantum state. The key difference from classical recurrence: quantum gates can create entanglement (correlations between qubits that have no classical analog) and interference (amplitudes can cancel or reinforce), enabling non-linear, global updates that additive SSMs cannot express.
-
Query-dependent measurement: In quantum mechanics, measurement collapses a superposition into a definite outcome. The measurement basis (which observable you measure) determines what information you extract. QLAM uses the current token’s query to choose the measurement basis, so different queries probe different aspects of the quantum state. This is like asking different questions of the same hologram—each question (measurement basis) reveals different features of the encoded history. The measurement yields classical values (expectation values of observables) that become the output for that timestep.
Framework Shift
Before (SSMs): After (QLAM):
Input x_t Input x_t
| |
v v
h_{t-1} --[Linear update]--> h_t |psi_{t-1}> --[PQC]--> |psi_t>
| | | |
v v v v
[Additive memory] Output y_t [Superposition] [Measurement]
|
v
Output y_t
Memory: vector of reals Memory: quantum amplitudes
Update: h_t = A*h_{t-1} + B*x_t Update: U(theta_t)|psi_{t-1}>
Capacity: O(d) states Capacity: O(2^d) states
Retrieval: linear readout Retrieval: query-dependent collapse
From additive recurrence to quantum evolution, the core shift is memory as superposition rather than accumulation—instead of summing contributions, QLAM encodes history as interfering probability amplitudes that can be selectively collapsed.
Expert Assessment
Problem choice: Real gap. The SSM vs attention tradeoff is well-established, and linear-time models with richer memory are a legitimate research direction. However, the paper positions itself as “one of the first” quantum sequence models, which feels like claiming novelty in a nascent subfield rather than solving a mature problem. The gap is real, but the solution space is still wide open.
Method maturity: Clever insight, but early-stage. Using quantum superposition for memory is conceptually elegant, but the paper doesn’t address practical quantum hardware constraints (noise, decoherence, gate fidelity). The experiments use simulated quantum circuits, which sidesteps the question of whether this works on real quantum computers. The PQC architecture seems somewhat arbitrary—why this gate sequence and not others? The paper would benefit from ablations showing which quantum features (entanglement, interference, measurement basis) actually matter.
Experimental integrity: Baselines are fair but limited. The paper compares against vanilla transformers, LSTMs, and S4/Mamba-style SSMs on sequential image tasks (sMNIST, sCIFAR-10). QLAM wins consistently, but the margins are modest (1-3% accuracy), and the tasks are toy problems where images are flattened into sequences—not natural language or real-world long-context tasks. The paper doesn’t report wall-clock time or quantum circuit depth, which are critical for assessing practical viability. No comparison to recent hybrid models like RWKV or RetNet that also aim for linear-time expressiveness.
Writing quality: The abstract and introduction are clear, but the method section is dense and assumes quantum computing background. The paper would benefit from a standalone “Quantum Primer” section explaining qubits, gates, and measurement for ML readers. The experimental section is thin—only 3 tasks, no error bars, no statistical significance tests. The related work section name-drops quantum ML papers but doesn’t clearly position QLAM relative to prior quantum sequence models (if any exist).
Verdict: weak accept — Interesting idea with solid intuition, but experiments are too limited and practical feasibility is unaddressed. This is a “proof of concept” paper that opens a research direction rather than a mature method ready for adoption.
Takeaways
For practitioners: Nothing immediately transferable unless you have access to quantum hardware or simulators. The core idea—using richer state representations to improve memory—is general, but the quantum implementation is too specialized.
For researchers: The measurement-as-retrieval framing is worth stealing. Instead of fixed readout heads, condition your retrieval mechanism on the query. This could apply to classical models: use query-dependent attention over a compressed memory bank, or learn query-conditioned projections of a latent state. The holographic memory metaphor also suggests a design principle: encode history as interference patterns (overlapping representations) rather than discrete slots, then use probing mechanisms to extract task-relevant features.
For quantum ML: This paper shows that quantum circuits can be integrated into recurrent architectures without breaking differentiability (gradients flow through measurement). The PQC-as-update-function pattern could generalize to other sequence tasks. However, the lack of noise analysis is a red flag—real quantum computers are noisy, and it’s unclear whether QLAM’s advantage survives decoherence.
论文: 2605.13833 作者: Hoang-Quan Nguyen, Sankalp Pandey, Khoa Luu 分类: cs.LG, cs.CV
缺口
Transformer 通过注意力机制建模长序列，但撞上了一堵墙：二次方复杂度意味着 10,000 个 token 在计算上变得不可行。
状态空间模型（SSM）如 S4 和 Mamba 用线性时间递归解决了成本问题，但它们的记忆更新是加性或线性的——每个新 token 只是被加到一个累加和上。
这限制了它们表示远距离 token 之间复杂非线性交互的能力。
缺口在于：我们有高效的记忆（SSM）或表达力强的记忆（注意力），但两者不可兼得。
SSM 通过 h_t = A·h_{t-1} + B·x_t 更新状态，这本质上是线性操作。
我们能否在不牺牲线性复杂度的前提下丰富状态表示？
问题：长序列需要全局上下文
|
v
约束：二次方注意力 vs 线性但受限的 SSM
|
+---> Transformer：O(n^2) 注意力
| |
| v
| 捕获全局交互但不可扩展
|
+---> SSM：O(n) 递归
|
v
可扩展但记忆 = 加性更新
|
v
假设：量子叠加态可以编码更丰富的状态
|
v
方法：QLAM - 量子态作为记忆，参数化电路作为更新
|
v
证据：在序列图像任务上击败 SSM 和 Transformer
|
v
结论：量子记忆 > 经典线性记忆用于序列建模
增量
一句话：QLAM 之前，序列模型在二次方表达力（注意力）和线性简洁性（SSM）之间二选一；QLAM 之后，量子叠加态提供了第三条路——线性时间加上非经典记忆容量。
核心机制
QLAM 用量子态替换 SSM 中的经典隐藏状态。
记忆不再是实数向量，而是一个量子态 |ψ⟩，其振幅编码了历史信息的叠加。
在每个时间步，一个参数化量子电路（PQC）作用于这个状态，以输入 token 为条件。
这是更新步骤——类似于经典递归中的 h_t = f(h_{t-1}, x_t)，但 f 现在是一个量子操作，可以创建纠缠和干涉。
为了提取信息，QLAM 使用依赖于查询的测量。
给定一个查询（例如来自当前 token），系统在由该查询决定的基下测量量子态。
测量使叠加态坍缩并产生经典输出，输入到下游层。
这个测量步骤是全局依赖浮现的地方：因为量子态经历了所有先前的 token，测量可以揭示加性更新会错过的相关性。
架构有三个阶段：(1) 将输入 token 编码为量子参数，(2) 应用 PQC 演化量子态，(3) 用依赖于查询的可观测量测量状态以产生输出。
关键是，步骤 1-3 按顺序发生，所以 n 个 token 的总成本是 O(n)——与 SSM 相同，但量子态可以表示比相同维度的经典向量多指数倍的配置。
输入 token x_t
|
v
[编码器] --> theta_t (量子参数)
|
v
|psi_{t-1}> --[PQC(theta_t)]--> |psi_t>
^ |
| v
+--- (量子态记忆) --------------+
|
v
查询 q_t --> [测量基]
|
v
可观测量 <O_q>
|
v
经典输出 y_t
把 QLAM 想象成全息胶片。
在经典 SSM 中，电影的每一帧通过在画布上添加墨水来存储——新信息覆盖或混合旧信息。
在 QLAM 中，每一帧被记录为全息胶片上的干涉图案。
当你用激光（查询）照射胶片时，干涉图案重建出 3D 图像——不是通过顺序读取像素，而是让所有重叠的波相互作用。
量子态就是全息图：它以叠加态保存所有过去的 token。
测量就是激光：它通过探测干涉图案提取与任务相关的信息。
PQC 是记录过程：它通过调制量子态的相位和振幅来编码每个新 token，创建编码 token 之间关系的相长和相消干涉。
关键概念
- 量子叠加态：经典比特是 0 或 1。
量子比特（qubit）可以处于状态 α|0⟩ + β|1⟩，其中 α 和 β 是满足 |α|² + |β|² = 1 的复数。
这意味着 qubit 在测量之前”既是 0 又是 1”。
对于 n 个 qubit，状态空间有 2^n 维——比 n 个经典比特大指数倍。
在 QLAM 中，这种指数容量让小的量子态能编码序列历史的丰富纠缠表示。
具体来说：10 个 qubit 可以同时表示 1024 个基态，而 10 个经典比特一次只能处于 1024 个状态中的一个。
- 参数化量子电路（PQC）：一系列量子门（旋转、纠缠操作），其角度是可学习的参数。
把它想象成神经网络层，但作用于量子态。
在 QLAM 中，PQC 接收当前量子态和输入 token，应用以 token 为条件的旋转，输出更新后的量子态。
与经典递归的关键区别：量子门可以创建纠缠（qubit 之间没有经典类比的相关性）和干涉（振幅可以抵消或增强），实现加性 SSM 无法表达的非线性全局更新。
- 依赖于查询的测量：在量子力学中，测量使叠加态坍缩为确定结果。
测量基（你测量哪个可观测量）决定你提取什么信息。
QLAM 使用当前 token 的查询来选择测量基，所以不同的查询探测量子态的不同方面。
这就像向同一个全息图提出不同的问题——每个问题（测量基）揭示编码历史的不同特征。
测量产生经典值（可观测量的期望值），成为该时间步的输出。
框架转变
之前（SSM）： 之后（QLAM）：
输入 x_t 输入 x_t
| |
v v
h_{t-1} --[线性更新]--> h_t |psi_{t-1}> --[PQC]--> |psi_t>
| | | |
v v v v
[加性记忆] 输出 y_t [叠加态] [测量]
|
v
输出 y_t
记忆：实数向量 记忆：量子振幅
更新：h_t = A*h_{t-1} + B*x_t 更新：U(theta_t)|psi_{t-1}>
容量：O(d) 个状态 容量：O(2^d) 个状态
检索：线性读出 检索：依赖于查询的坍缩
从加性递归到量子演化，核心转变是记忆作为叠加而非累积——不是求和贡献，QLAM 将历史编码为可以选择性坍缩的干涉概率振幅。
专家评审
选题眼光：真实缺口。
SSM 与注意力的权衡已被充分确立，具有更丰富记忆的线性时间模型是合理的研究方向。
然而，论文将自己定位为”最早的”量子序列模型之一，这感觉像是在新兴子领域声称新颖性，而不是解决成熟问题。
缺口是真实的，但解决方案空间仍然很开放。
方法成熟度：巧妙洞察，但处于早期阶段。
使用量子叠加态作为记忆在概念上很优雅，但论文没有解决实际量子硬件约束（噪声、退相干、门保真度）。
实验使用模拟量子电路，这回避了这在真实量子计算机上是否有效的问题。
PQC 架构似乎有些随意——为什么是这个门序列而不是其他？论文将受益于消融实验，显示哪些量子特征（纠缠、干涉、测量基）真正重要。
实验诚意：基线公平但有限。
论文在序列图像任务（sMNIST、sCIFAR-10）上与原始 Transformer、LSTM 和 S4/Mamba 风格的 SSM 进行比较。
QLAM 始终获胜，但优势适中（1-3% 准确率），任务是玩具问题，图像被展平为序列——不是自然语言或真实世界的长上下文任务。
论文没有报告实际运行时间或量子电路深度，这对评估实际可行性至关重要。
没有与最近的混合模型（如 RWKV 或 RetNet）进行比较，这些模型也旨在实现线性时间表达力。
写作功力：摘要和引言清晰，但方法部分密集，假设读者有量子计算背景。
论文将受益于独立的”量子入门”部分，为 ML 读者解释 qubit、门和测量。
实验部分单薄——只有 3 个任务，没有误差条，没有统计显著性检验。
相关工作部分提到量子 ML 论文，但没有清楚地将 QLAM 相对于先前的量子序列模型（如果存在）定位。
判决：弱接收 — 有趣的想法，直觉扎实，但实验太有限，实际可行性未得到解决。
这是一篇”概念验证”论文，开辟了研究方向，而不是准备好采用的成熟方法。
要点总结
对实践者：除非你能访问量子硬件或模拟器，否则没有立即可迁移的东西。
核心思想——使用更丰富的状态表示来改善记忆——是通用的，但量子实现太专业化了。
对研究者：测量即检索的框架值得借鉴。
不要使用固定的读出头，而是根据查询调节检索机制。
这可以应用于经典模型：对压缩记忆库使用依赖于查询的注意力，或学习潜在状态的依赖于查询的投影。
全息记忆隐喻也提示了一个设计原则：将历史编码为干涉图案（重叠表示）而不是离散槽，然后使用探测机制提取与任务相关的特征。
对量子 ML：本文表明量子电路可以集成到递归架构中而不破坏可微性（梯度通过测量流动）。
PQC 作为更新函数的模式可以推广到其他序列任务。
然而，缺乏噪声分析是一个危险信号——真实的量子计算机是有噪声的，QLAM 的优势是否能在退相干中幸存尚不清楚。