SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal LLMs

Paper: 2604.19638 Authors: Josue Torres-Fonseca, Naihao Deng, et al. (University of Michigan, Boise State) Categories: cs.AI, cs.CV

Problem

Multimodal LLMs are increasingly adopted as autonomous agents in interactive environments, yet their ability to proactively address safety hazards remains insufficient.

Existing safety benchmarks (ASIMOV, MM-SafetyBench) focus on hazard recognition through question-answering on static images/videos - a critical gap remains in evaluating action-oriented safety.

SafetyALFRED Benchmark

Built on ALFRED embodied agent benchmark, augmented with six categories of real-world kitchen hazards:

1. Sharp object handling
2. Hot surface awareness
3. Liquid spillage prevention
4. Chemical safety
5. Fall prevention
6. Electrical hazards

Key Finding: Alignment Gap

Evaluated 11 state-of-the-art models (Qwen, Gemma, Gemini families):

• Models can accurately recognize hazards in QA settings
• Average mitigation success rates are low in embodied planning contexts

Static QA evaluations are insufficient for physical safety because they don’t test whether models can take corrective actions.

Takeaways

• Need paradigm shift toward benchmarks that prioritize corrective actions in embodied contexts
• Hazard recognition ≠ Hazard mitigation
• Future safety evaluation must test action, not just perception