Decoding-Level Taboo: Kicking Models Off Their Comfortable Path

Paper: 2608.09900 Authors: Tadanobu Chuyo Kamijo, Ori Rottenstreich, Javier Conde, Gonzalo Martínez, Pedro Reviriego Categories: cs.CL

The Gap

The honest version of the gap is this: nearly every LLM benchmark measures the model doing the thing it most wants to do. You give it a prompt, it greedily (or near-greedily) follows the highest-probability path, and we score the output. That path is the most heavily optimized region of the model’s behavior — it’s what pretraining compressed toward and what RLHF polished. Call it the nominal corridor.

Deployment doesn’t look like that. A production system has a 2000-token system prompt, a JSON schema it must satisfy, a safety filter that vetoes certain phrasings, a style guide that bans certain words, a retrieval context that contradicts the model’s priors. All of these are, functionally, constraints that push generation off the highest-probability path. And the field’s existing tools for measuring what happens then are surprisingly weak:

• Adversarial prompting / jailbreak suites (AdvBench-style, GCG, PAIR): they measure whether a constraint can be *broken, not whether the model degrades gracefully when constraints hold. They also confound prompt sensitivity with the thing being measured — a bad result might just mean the prompt was awkward.
• Constrained decoding work (grammar-constrained decoding, Outlines/JSONFormer, NeuroLogic, lexically constrained MT): treats off-path generation as an *engineering problem to solve, and reports whether the constraint was satisfied. It doesn’t use the constraint as a measuring instrument.
• Robustness benchmarks via input perturbation (typos, paraphrase, distractors): perturb the input, but the model still gets to walk its favorite output path afterward. The corridor stays intact.
• Scaling-law and instruction-tuning evaluations: report nominal-condition curves. They tell you nothing about whether alignment buys graceful degradation or just polished nominal behavior.

So the specific boundary: nobody has a cheap, prompt-free, dial-adjustable way to force a model off its nominal path and watch what breaks. That’s the hole Taboo fills — by intervening in logit space at runtime rather than in the prompt.

  [PROBLEM]
  benchmark scores measured only inside the
  nominal generation corridor
        |
        v
  [OBSERVATION]
  real deployments (system prompts, guardrails,
  schemas) constantly push generation off-corridor
        |
        v
  [ASSUMPTION]
  off-path behavior is a distinct, measurable
  capability -- not predictable from on-path scores
        |
        v
  [METHOD] Decoding-Level Taboo
  at each word boundary: mask top candidate token(s)
  in logit space  =>  force circumlocution
  (no prompt text involved: "zero-prompt")
        |
        +------------------------+
        v                        v
  [EVIDENCE 1]             [EVIDENCE 2]
  robustness scales        robustness improves
  with parameter count     with instruction
                           alignment / post-training
        |                        |
        +------------+-----------+
                     v
  [CONCLUSION]
  off-path robustness is a real, separable axis;
  Taboo is a reusable primitive for auditing it
  (+ synthetic data, + guardrail stress tests)

The Increment

One sentence: Before this paper, “can the model still function when it can’t say the obvious thing?” was a vibe you formed from anecdotes; after it, it’s a knob you can turn at inference time on any open-weight model without touching the prompt.

Core Mechanism

The mechanism is almost embarrassingly simple, which is a compliment. At each decoding step the model produces a logit vector over the vocabulary. Normally you sample or argmax from it. Taboo inserts a step in between: at positions that begin a new word, it identifies the top-ranked candidate token (or top-k) and sets its logit to negative infinity. The model then must pick its second-best (or (k+1)-th) option and continue. Crucially the mask is applied *dynamically — it’s not a fixed banned-word list computed in advance, it’s whatever the model happened to want most at that exact position, given everything it has already generated.

The word-boundary condition is the load-bearing design detail, and it’s worth dwelling on. If you masked the top token at every step, you’d break words in half: the model commits to “circum”, you ban “vent”, it emits “cumference” and you get lexical garbage. That measures tokenizer brittleness, not reasoning. By gating the intervention to word-initial positions (in most modern BPE tokenizers, tokens carrying a leading-space marker), the model is forced to choose a different word, then allowed to spell that word out unimpeded. The unit of forced divergence becomes semantic rather than orthographic. That’s what turns the intervention into a probe of circumlocution ability — can you say the thing another way? — rather than a probe of spelling.

Data flow, then: prompt goes in unmodified (this is the “zero-prompt” property — no instruction like “don’t use the word X”, so no confound from instruction-following ability, no prompt-length effects, no way for the model to refuse or negotiate). The logits processor sits between the forward pass and the sampler. It maintains just enough state to know whether the next token starts a word. Severity is a dial: k = 1 is a nudge, k = 5 is a shove, k = 20 is closer to a wall. You then measure the resulting text along whatever axes you care about — task success, coherence, factuality, instruction compliance — and compare against the unmasked run of the same model on the same prompt. The delta is the robustness score.

  PROMPT (unmodified)
        |
        v
  +-------------------------------+
  |        LLM forward pass       |
  +-------------------------------+
        |
        v  logits over vocabulary V
        |
  +-------------------------------+
  |   TABOO LOGITS PROCESSOR      |
  |                               |
  |  is next token word-initial?  |
  |        |            |         |
  |       yes           no        |
  |        |            |         |
  |        v            v         |
  |  rank top-k    pass through   |
  |  set those      unchanged     |
  |  logits = -inf                |
  +-------------------------------+
        |
        v  masked logits
  +-------------------------------+
  |     sampler / argmax          |
  +-------------------------------+
        |
        v
  token ..... appended to context
        |
        +--------> loop back (mask is recomputed
                   from the NEW context, not fixed)

  OUTPUT: circumlocuted text
        |
        v
  compare vs unmasked run  =>  degradation profile
                               as function of k

The structural metaphor: rolling roadblocks in a city.

Think of generation as driving to an address. The logit distribution at each step is the set of roads leaving the current intersection, ranked by how much the driver wants to take each one. Normal decoding is a local who takes their habitual route every time — the route they’ve driven ten thousand times.

Taboo is a traffic crew that follows the car. At every intersection (that’s the word boundary — note they do *not set up cones mid-block, which would just cause a crash, not a detour) they close the road the driver was about to take (the top logit). Not a pre-published list of closed streets — that would be a fixed banned-word list, and the driver could plan around it. This is a cone appearing the instant the turn signal goes on. The number of lanes they close is k: close one road and a good driver shrugs and takes the parallel street; close the five best roads and now you need to actually know the city.

What you learn is exactly what you’d learn watching different drivers under this harassment. The tourist with no mental map circles, panics, ends up in the wrong district, or starts babbling about a completely different destination — that’s a small model losing coherence and hallucinating. The cab driver with twenty years of the city in their head takes a longer, uglier, slightly slower route and still arrives — that’s the large aligned model paraphrasing its way to the same answer. And the paper’s second finding maps cleanly too: instruction alignment is like a driver who has internalized “the passenger’s destination is the goal,” so when the road closes they re-route toward the destination instead of toward the road they liked. An unaligned base model is more like a driver following a groove in the asphalt — remove the groove, and there’s no destination-level commitment left to fall back on.

The metaphor also predicts the method’s main weakness, which is a good sign for a metaphor: closing roads tells you about the driver’s redundancy of knowledge, but it doesn’t tell you whether real-world traffic looks anything like rolling roadblocks. More on that below.

Key Concepts

• Logits, and why masking one is a surgical act: Before an LLM emits a word, it produces one number per vocabulary entry — maybe 128,000 numbers. Higher number, more preferred. Softmax turns them into probabilities. The important intuition is that this vector is a *complete ranked preference list, not a single choice, and it’s freshly recomputed at every position from the entire context. So if you set the top entry to negative infinity, you haven’t damaged the model, confused it, or told it anything. It doesn’t “know” it was blocked. It simply acts on its second preference, and then — this is the subtle part — the next step’s preferences are computed from a context that now contains that second-best word. The model rationalizes forward from wherever it lands. Concretely: ask it about the ocean, it wants “The ocean is vast”; ban “vast” and you get “The ocean is enormous”; ban both and maybe “The ocean covers”; ban the top five and you might get “The ocean, in terms of its sheer scale…”. Each step is locally sensible; the question is whether the trajectory still lands on truth.

• Circumlocution as a capability, not a failure mode: In humans, talking around a word is a sign of *strong language ability — it’s what a fluent speaker does when a word is on the tip of the tongue, and it’s what aphasia patients lose. The paper’s clever reframe is to treat forced circumlocution as a measurement. A model that can only express a fact one way has memorized a surface form; a model that can express it five ways has something closer to a representation of the fact. Concrete test: the answer to “who wrote Hamlet” can be “Shakespeare,” “the Bard of Avon,” “an English playwright born in Stratford in 1564.” A system that collapses to nonsense once “Shakespeare” is banned never really knew the answer in a robust sense — it knew a string.

• Zero-prompt intervention (and why “zero-prompt” is the actual contribution): If you wanted to test the same thing with a prompt — “answer this without using the words X, Y, Z” — you’d immediately contaminate the experiment with three other abilities: understanding the negative instruction, tracking a constraint over long outputs, and the general instruction-following gap between models. Small models fail such prompts because they can’t follow instructions, not because they lack lexical flexibility. Working in logit space sidesteps all of it. The constraint is *enforced by construction — 100% compliance, identically, for a 1B base model and a 70B chat model. That’s what makes cross-model comparison meaningful, and it’s the piece of this paper I’d actually reuse.

Framework Shift

  Before (mainstream approach):          After (this paper):

  PROMPT LAYER is the test surface       DECODING LAYER is the test surface

  +---------------------------+          +---------------------------+
  | "answer without using X"  |          |  prompt: unchanged        |
  | adversarial suffix        |          +---------------------------+
  | typos / paraphrase        |                    |
  +---------------------------+                    v
            |                             +---------------------+
            v                             | logits              |
  +---------------------------+            |  [ban top-k at      |
  | model may: comply,        |            |   word boundary]    |
  |   ignore, refuse,         |            +---------------------+
  |   partially comply        |                    |
  +---------------------------+                    v
            |                             constraint holds
            v                             100% for every model
  model still free to walk its            model FORCED off corridor
  favorite output path                    at every single word

  result: measures                        result: measures
    instruction-following                   off-path robustness
    + robustness, tangled                   in isolation

  knob: none (binary prompts)              knob: k = severity dial
                                                  (continuous stress curve)

One sentence: From *asking a model to leave its comfort zone and hoping it complies, to removing the comfort zone from underneath it — the core shift is moving the experimental control from the prompt into the decoder, which converts a confounded request into an enforced, dial-adjustable constraint.

Expert Assessment

Problem choice: Real gap, and well-chosen. The benchmark-vs-deployment divergence is one of the field’s genuinely unsolved measurement problems, and “we only ever evaluate on the optimized path” is a sharp, non-obvious articulation of why. I also like that the intervention is *orthogonal to existing eval axes — it’s not another dataset, it’s a lens you can put in front of any dataset. That’s the kind of contribution that ages well. The weak spot in the framing is the leap from “system prompts and guardrails push models off-path” to “therefore banning top tokens at word boundaries is a good proxy for that.” Those are different geometries of constraint. A JSON schema restricts structure globally; a safety filter restricts a semantic region; Taboo restricts whatever the model wanted most, everywhere, uniformly. It’s plausible these correlate, but plausible is doing a lot of work in the motivation. The paper would be dramatically stronger with even one experiment showing Taboo scores predict degradation under a real deployment constraint. Without that, this is a nice stress test in search of external validity.

Method maturity: Clever framing, not clever machinery. The mechanism is a logits processor — HuggingFace has shipped NoBadWordsLogitsProcessor for years, banned-token decoding is standard in constrained-generation work, and “set logit to -inf” is the first thing anyone tries. The genuine insight, and it’s worth something, is the word-boundary gate: that’s the difference between measuring semantics and measuring tokenizer damage, and I’d bet a lot of naive replications get this wrong. The zero-prompt argument is also a real methodological point, cleanly made. But several simpler or more informative variants seem unexplored: masking a *random non-top token (essential control — how much of the degradation is “off-path” versus just “lower-probability text”?), masking the top token only at content words versus function words (banning “the” is a very different test than banning “photosynthesis”), and comparing against a straight temperature/top-p sweep, which is the cheapest possible off-path intervention and a baseline this method must beat to justify itself. Also unaddressed as far as I can tell: entropy confound. At high-entropy positions the top token is barely preferred and masking costs nothing; at low-entropy positions it’s devastating. Since better-calibrated larger models have systematically different entropy profiles, “robustness improves with scale” may be partly a restatement of “larger models are less peaked in different places.” That needs controlling for, ideally by matching interventions on probability mass removed rather than rank.

Experimental integrity: This is where I’d push hardest, and the abstract makes it hard to be generous. The reported findings — robustness improves with model size, and with instruction alignment — are directionally exactly what everyone would predict before reading the paper. That’s not disqualifying (confirming an expectation with a new instrument is legitimate), but it means the paper’s value rests almost entirely on the *instrument’s validity, and no numbers, no error bars, and no metric definition appear in the abstract. Two specific red flags to check in the body. First, how is degradation scored? If it’s an LLM judge, we have circularity risk: judges systematically penalize unusual phrasing, and Taboo’s whole purpose is to produce unusual phrasing, so “coherence drops” may partly measure judge distaste. If it’s perplexity under a reference model, the result is near-tautological — you removed the highest-probability tokens, of course perplexity went up. The only fully safe measurements are task-success on verifiable tasks (arithmetic, closed-book QA with exact match, code that compiles). Second, is scale confounded with family? Comparing across “several open-weight families” while varying both size and alignment invites the classic trap where a family-level training-data difference gets read as a scale effect. The clean design is within-family size ladders plus base-versus-instruct pairs at matched size (Llama, Qwen, and Pythia all support this) — I hope that’s what was done, but the abstract doesn’t say.

Writing quality: The abstract is written with real rhetorical skill — “a narrow, highly optimized generation corridor” is a phrase I’ll steal — but it also shows where the corners were cut. The last sentence is a wish list (“synthetic datasets, guardrail stress-testing, pre-deployment auditing”) presented as contribution, and that’s the tell of a paper whose results section is thinner than its ambitions. Any one of those three, demonstrated even minimally, would be worth more than all three asserted. The section that would most elevate the paper is the metric and validity section: define degradation precisely, include the random-token-masking and temperature-sweep controls, and add one experiment linking Taboo scores to behavior under an actual production-style constraint. Do that and this goes from an interesting probe to a tool people cite. Also worth trimming: the framing occasionally implies Taboo simulates guardrails, which overclaims — it’s a *generic off-path stressor, and saying so plainly would cost the paper nothing and buy it credibility.

Verdict: borderline, leaning weak accept as a short/workshop paper — a genuinely useful and reusable primitive with an under-argued validity story and findings that confirm rather than surprise. I’d accept it at a workshop tomorrow and send it back from a main track with a request for the controls above.

Takeaways

Concrete things worth stealing:

1. Move your experimental control out of the prompt whenever you can. This is the transferable methodological lesson and it generalizes far beyond taboo words. Any time you test a model by *asking it to do something unusual, your measurement is contaminated by instruction-following ability — which is precisely the variable that differs most across model sizes and alignment stages. Enforcing the constraint in the decoder gives you 100% compliance by construction, making cross-model comparison honest. The same trick applies to testing format adherence, length control, and refusal behavior: enforce, don’t request.

2. The word-boundary gate is a reusable primitive. If you ever intervene on token distributions and care about semantics rather than orthography, gate your intervention to word-initial positions. Otherwise you’re measuring your tokenizer. This tiny detail is the difference between a signal and noise, and it applies to token-level attribution, token dropout, and noise-injection robustness studies too.

3. Forced paraphrase as a knowledge test. Practical use today, independent of this paper’s results: to check whether a model *knows a fact versus memorized a string, ban the canonical surface form and see if the answer survives. This is a cheap, judge-free probe for contamination and shallow memorization — the answer either still comes out correct or it doesn’t.

4. Taboo as a synthetic-data generator, which is arguably the most immediately valuable use. Running the same prompt at k = 1, 3, 5, 10 gives you a family of semantically-similar, lexically-divergent outputs from a single model, for free, with no prompt engineering and no sampling-temperature degeneracy. That’s a paraphrase corpus, a hard-negatives set for retrievers and rerankers, and a diversity-injection mechanism for distillation data. I’d expect this application to outlive the diagnostic framing.

5. A cheap red-team addition, with a caveat. Guardrails tuned on nominal phrasings may miss circumlocuted versions of the same content, and Taboo generates those mechanically. Worth adding to a safety eval pipeline. The caveat: this is a defensive audit of your own system, and unlike prompt-based attacks it requires logit access, so it’s a tool for model owners, not an externally exploitable vector.

What not to take: don’t treat “robustness under Taboo” as a validated predictor of deployment performance yet. The paper asserts the connection more than it demonstrates it. Use Taboo as a diagnostic that generates hypotheses, then confirm on the constraint you actually care about.