UMPIRE: Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume

Paper: 2602.24195 Authors: Gregory Kang Ruey Lau, Hieu Dao, Nicole Kan Hui Lin, Bryan Kian Hsiang Low Categories: cs.AI, cs.CL, cs.CV, cs.LG

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities across diverse tasks, yet they remain prone to generating plausible but incorrect outputs—a critical barrier to reliable deployment in high-stakes applications. This paper introduces UMPIRE (Uncertainty quantification for Multimodal models using incoherence-adjusted semantic volume with internal modality features), a novel training-free framework for uncertainty quantification that addresses key limitations of existing approaches. Unlike prior methods constrained to specific modalities, dependent on external tools, or computationally prohibitive, UMPIRE operates efficiently across various input and output modalities using only the model’s internal features. The framework computes an incoherence-adjusted semantic volume of sampled responses, capturing both global semantic diversity and local response incoherence based on internal model confidence signals.

Key Contributions

• A training-free, modality-agnostic uncertainty quantification framework that works across image, audio, video, and text modalities without requiring external tools or model modifications
• Introduction of incoherence-adjusted semantic volume as a unified metric that combines global semantic diversity with local response coherence based on internal model confidence
• Theoretical analysis establishing uncertainty desiderata for MLLMs and providing formal justification for UMPIRE’s design principles
• Comprehensive experimental validation demonstrating consistent superiority over baseline methods in error detection and uncertainty calibration across multiple benchmarks
• Demonstration of generalization to non-text generation tasks including image and audio synthesis

Methodology: Incoherence-adjusted Semantic Volume

UMPIRE’s core innovation lies in its computation of uncertainty through two complementary components. The framework first samples multiple responses from the MLLM for a given input, then extracts internal modality-specific features (e.g., vision encoder outputs for images, audio encoder outputs for speech).

The semantic volume component measures the global diversity of sampled responses by computing the volume of the convex hull formed by their feature representations in the model’s latent space. This captures the spread of possible interpretations the model considers plausible. Mathematically, for a set of $n$ sampled responses with feature vectors $\{f_1, f_2, ..., f_n\}$, the semantic volume $V$ quantifies the geometric extent of the response distribution.

The incoherence adjustment component leverages the model’s internal confidence signals to identify locally inconsistent or uncertain responses. By analyzing attention patterns, token probabilities, or layer-wise activations, UMPIRE detects responses where the model exhibits internal disagreement. This local incoherence metric $I$ is then used to adjust the semantic volume, yielding the final uncertainty score:

$U = V \cdot (1 + \alpha \cdot I)$

where $\alpha$ is a scaling parameter. This formulation ensures that uncertainty increases both when the model produces diverse responses (high $V$) and when individual responses show internal inconsistency (high $I$).

Experimental Results and Performance

The authors conducted extensive experiments across multiple benchmarks spanning different modalities and task types. For vision-language tasks, UMPIRE was evaluated on VQAv2, GQA, and POPE datasets, demonstrating consistent improvements in AUROC (Area Under Receiver Operating Characteristic) for error detection compared to baseline methods including entropy-based metrics, semantic entropy, and lexical similarity approaches.

In adversarial settings using datasets like ADVQA and CV-Bench, UMPIRE maintained robust performance while baseline methods degraded significantly. The framework achieved particularly strong results in out-of-distribution scenarios, where the incoherence adjustment proved crucial for identifying model uncertainty on unfamiliar inputs.

For audio-text tasks using datasets like Clotho-AQA and AudioCaps, UMPIRE outperformed modality-specific baselines by 8-15% in error detection metrics. The framework’s ability to leverage internal audio encoder features without requiring specialized audio processing tools demonstrated its practical advantages.

Video understanding experiments on NExT-QA and MSRVTT-QA showed that UMPIRE’s uncertainty estimates correlated strongly with actual model errors, enabling effective selective prediction where low-confidence queries could be escalated to human experts or more capable models.

Implications for Reliable AI Deployment

UMPIRE addresses a critical gap in making MLLMs production-ready for high-stakes applications. The framework’s training-free nature means it can be immediately applied to existing models without requiring additional data collection, fine-tuning, or architectural modifications. This practical advantage is particularly valuable given the rapid pace of MLLM development.

The ability to quantify uncertainty across modalities opens new possibilities for hybrid human-AI systems. By identifying queries where the model is uncertain, UMPIRE enables intelligent routing strategies: high-confidence predictions can be automated, while uncertain cases are escalated for human review or processed by more capable (but expensive) models. This selective prediction paradigm can significantly improve both accuracy and cost-efficiency.

The framework’s success in adversarial and out-of-distribution settings suggests it could serve as a valuable safety mechanism, helping detect when MLLMs encounter inputs outside their training distribution or potentially adversarial examples designed to elicit incorrect responses.

Takeaways

1. UMPIRE provides a practical, training-free solution for uncertainty quantification in MLLMs that works across multiple input and output modalities without external dependencies.

2. The incoherence-adjusted semantic volume metric effectively combines global response diversity with local model confidence signals, outperforming existing uncertainty quantification approaches.

3. Extensive experiments demonstrate consistent improvements in error detection (8-15% AUROC gains) and uncertainty calibration across vision, audio, and video benchmarks.

4. The framework maintains robust performance in adversarial and out-of-distribution settings where baseline methods degrade significantly.

5. UMPIRE’s modality-agnostic design enables generalization to non-text generation tasks including image and audio synthesis, expanding the scope of reliable uncertainty quantification.

6. The training-free nature and reliance on internal model features make UMPIRE immediately applicable to existing MLLMs, facilitating practical deployment in production systems.

7. By enabling selective prediction and intelligent query routing, UMPIRE provides a pathway toward more reliable and cost-effective deployment of MLLMs in high-stakes applications.