When Robots Mishear Us: ASR Errors Are an Embodied Safety Problem

Paper: 2608.28518 Authors: Sihan Jia, Oliver Lemon Categories: cs.AI, cs.CL, cs.RO

The Gap

Safety evaluation for embodied AI models assumes, mostly implicitly, that the instruction the model receives is the instruction the user issued. Voice-controlled embodied agents break that assumption at the front of the pipeline: an automatic speech recognition system transcribes what the user said, and that transcript — not the user’s intent — is what the safety mechanism sees.

The consequence is a safety surface that exists before any safety mechanism runs, and it has been unexamined. A model that reliably refuses a harmful request can still execute that request if the refusal machinery never receives it intact. And the failure is not random noise: speech recognition errors are structured — they cluster on particular phonemes, on proper nouns, on words that sound like other words — so the errors that reach the model are a systematically distorted version of user intent, not an unbiased one.

   THE SAFETY PIPELINE AND WHERE IT IS ASSUMED TO START

   user speaks
        |
        v
   [ ASR ] -- transcribes -->  transcript
        |                          |
        |                          v
        |                    [ safety mechanism + model ]
        |                          |
        |                          v
        |                       action executed
        |
   ASSUMPTION (usually implicit): transcript == what the user meant
        |
        v
   [GAP] the safety surface begins BEFORE any safety mechanism runs
        |
        +-- a model that reliably refuses a harmful request can
        |   still execute it if the request never arrives intact
        |
        +-- ASR errors are STRUCTURED, not random noise:
              they cluster on phonemes, proper nouns, and
              words that sound like other words
           -> what reaches the model is a SYSTEMATICALLY
              distorted version of intent, not an unbiased one

The Increment

One sentence: Before this paper, embodied AI safety was evaluated on correctly transcribed instructions; after it, simulated ASR errors combined with existing safety benchmarks show that mishearing produces executed harmful instructions, via two distinguishable mechanisms.

Core Mechanism

The method is a composition rather than a new benchmark: simulate ASR errors, then feed them into existing safety benchmarks — SafeAgentBench and POEX — to measure how different classes of error affect embodied safety. Building on established benchmarks is the right choice here, because it isolates the new variable: the difference between results is attributable to the transcription error, not to a new evaluation harness.

The central finding is that mishearing produces harmful instructions being accepted and executed, so the safety property degrades. And the degradation is not uniform — it comes in two mechanisms that the paper separates:

Some errors preserve semantic structure but increase harmful ambiguity. The transcription remains a coherent instruction; it just admits a harmful reading it did not have before. This is the more insidious class, because nothing looks broken. A safety mechanism scanning for a specific harmful keyword finds no keyword; what changed is that the request is now ambiguous in a direction that permits harm. Ambiguity is a well-known failure mode for safety classifiers, and the paper’s contribution is showing that speech recognition errors are a systematic source of it.

Others weaken the model’s refusal behaviour, allowing unsafe plans to be generated and executed. This is a different mechanism: it is not that the request became ambiguous, but that the model’s disposition to refuse was itself degraded. A weakened refusal is closer to the failure mode people worry about with jailbreaks, arriving here through an accidental route.

Then the result that makes this an engineering problem rather than only a risk finding: automatic correction of ASR errors can reduce the risk in some cases, but this is not always effective. A partial mitigation is arguably the most useful thing to report, because it prevents the obvious conclusion on either side. One cannot say “fix the ASR and the problem is solved” — the fix helps sometimes. Nor can one say the problem is unsolvable. What it means is that ASR correction is a mitigation with a measurable hit rate, and treating it as a complete solution would be a mistake.

   METHOD: COMPOSE, DON'T REPLACE

   simulate ASR errors
        |
        v
   feed into EXISTING safety benchmarks
     (SafeAgentBench, POEX)
        |
        v
   isolates the new variable: differences are attributable
   to the TRANSCRIPTION ERROR, not to a new harness

   FINDING: harmful instructions get ACCEPTED AND EXECUTED
        |
        +-- MECHANISM 1: preserved semantic structure,
        |                INCREASED HARMFUL AMBIGUITY
        |     the transcript is still a coherent instruction
        |     -> but now admits a harmful reading
        |     -> nothing looks broken; no harmful keyword to find
        |
        +-- MECHANISM 2: WEAKENED REFUSAL BEHAVIOUR
                         -> unsafe plans generated and executed
              not that the request became ambiguous, but that
              the disposition to refuse was itself degraded

   MITIGATION: automatic ASR correction helps SOMETIMES
        -> not a complete fix, and not useless either
        -> it is a mitigation with a measurable hit rate

Think of it as a locked door with a bad intercom. The lock is excellent and the guard is well trained. But the intercom garbles names: “let me in, I’m the plumber” can arrive as “let me in, I’m a plumber” or, on a bad line, as something the guard parses as an existing resident. The lock did nothing wrong and the guard did nothing wrong; the vulnerability was created one step earlier, at the point where intent became text. And the two mechanisms have analogues here: sometimes the garbled message is still a fluent sentence that now happens to authorise entry (ambiguity), and sometimes the guard just becomes less sure of the rules and waves people through (weakened refusal). Fitting a better intercom helps, and it does not close the class of problems, because the next bad line will garble something else.

Key Concepts

• The safety surface starts before the safety mechanism: ASR output is what the model is evaluated on, so transcription errors are inside the threat model rather than outside it. It relocates a fault line that most evaluations assume is upstream.
• Structured, not random, error: speech recognition mistakes cluster on phonemes, proper nouns and homophone-adjacent words. A systematically biased input distribution is worse for safety than random noise, because the failures concentrate rather than averaging out.
• Two mechanisms, not one: increased harmful ambiguity versus weakened refusal. They call for different detection strategies — the first needs ambiguity-aware checking rather than keyword scanning, the second needs monitoring of refusal behaviour itself.
• Partial mitigation as the honest result: ASR correction helps in some cases and not others. Reporting it as partial is more useful than either extreme, because it defines the residual risk that must be handled elsewhere.

Framework Shift

Before (safety evaluated on correct transcripts):
  assume the instruction received == the instruction issued
  -> test the model on harmful requests, measure refusal
  -> the transcription step is outside the evaluation

After (transcription error inside the threat model):
  simulate ASR errors, feed into existing safety benchmarks
  -> harmful instructions ACCEPTED AND EXECUTED
  -> two mechanisms: harmful ambiguity | weakened refusal
  -> automatic correction helps sometimes, not always

From evaluating whether a model refuses harmful requests, to evaluating whether a harmful request survives the journey into the model, the core shift is that the pipeline’s front end is part of the safety boundary.

Expert Assessment

Problem choice: Excellent, and it identifies a component that safety evaluations had been treating as transparent. The insight is simple once stated — the model never sees the user’s intent, only a transcript — and it reframes a class of failures as an input-integrity problem rather than a model-alignment one.

Method maturity: Composing with existing benchmarks rather than building a new one is the right methodological call, because it isolates the transcription variable and makes the results comparable to prior embodied-safety work. Separating the two mechanisms is the substantive analytical contribution: harmful ambiguity and weakened refusal are different failures with different signatures, and distinguishing them is what lets a defender choose a response rather than just knowing there is a problem.

Experimental integrity: Simulating ASR errors rather than collecting real misrecognitions is a reasonable choice for a first study — it gives control over the error classes — but it also means the results describe the simulated error distribution, and the paper is appropriately careful not to claim a real-world rate. The partial-mitigation result is the most trustworthy kind of finding because it is inconvenient for both narratives. The main limitation is scope: the mechanism is studied on embodied benchmarks, and the extent to which the same two failure modes appear in non-embodied voice interfaces is left open.

Writing quality: The two-mechanism split is the clearest part of the exposition and is worth the space it takes. Because this is a safety result aimed partly at practitioners, a short section on what to monitor — ambiguity-aware checking rather than keyword filters, and refusal-rate tracking as a first-class signal — would make it considerably more actionable.

Verdict: accept — a well-scoped demonstration that a pipeline component treated as transparent is a systematic source of safety failures, with the failures characterised well enough to act on.

Takeaways

• Put the front of your pipeline inside the threat model. If a model is evaluated on a transcript, transcription errors are part of the safety surface, not a preprocessing detail.
• Treat systematically biased input as worse than random noise. ASR errors cluster, so the failures concentrate on particular words instead of averaging out.
• Separate ambiguity failures from refusal failures and monitor them differently. Keyword scanning will not catch a request that became harmful by becoming vague; refusal-rate tracking will.
• Report partial mitigations as partial. If automatic correction fixes some cases and not others, that hit rate is the residual risk someone else has to handle.