MuRF: Multi-Resolution Fusion for Vision Foundation Models

Paper: 2603.25744 Authors: Bocheng Zou, Mu Cai, Mark Stanley, Dingfu Lu, Yong Jae Lee Categories: cs.CV

The Gap

Vision Foundation Models (VFMs) like DINOv2 have reached a point where they can handle variable input sizes during training, but inference remains stuck at a single fixed scale. This is wasteful because visual perception fundamentally operates across scales: low-res captures global semantics (what’s in the scene), high-res captures fine details (where exactly, what texture). Prior work either trains scale-specific models (expensive, inflexible) or uses single-scale inference (leaves information on the table). The gap: we have models that could theoretically see at multiple scales, but no one’s figured out how to fuse those views at inference without retraining.

Problem: Single-scale inference wastes complementary information
   |
   v
Assumption: Different resolutions = different inductive biases
   |
   v
Method: Process same image at multiple scales, fuse features
   |
   v
Evidence: Consistent gains across tasks (segmentation, depth, retrieval)
   |
   v
Conclusion: Multi-scale fusion is a universal, training-free enhancement

The Increment

One sentence: Before MuRF, you picked one resolution and lived with its trade-offs; after MuRF, you get the best of all resolutions simultaneously without retraining.

Core Mechanism

MuRF takes a frozen VFM and feeds it the same image at multiple resolutions (e.g., 224px, 448px, 672px). Each resolution produces a feature map. The low-res map captures global context efficiently; the high-res map captures fine-grained details but at higher computational cost. MuRF then upsamples all feature maps to a common spatial resolution and fuses them—either by simple averaging or learned lightweight fusion modules (cross-attention or MLP-based).

The key insight: the VFM’s weights stay frozen. No backpropagation through the backbone. The fusion happens in feature space, not pixel space. This means you can drop MuRF onto any existing VFM (DINOv2, SigLIP2, etc.) and immediately get multi-scale reasoning without touching the pretrained weights.

Input Image
    |
    +----> [Resize: 224px] --> VFM (frozen) --> Feature Map A (low-res)
    |
    +----> [Resize: 448px] --> VFM (frozen) --> Feature Map B (mid-res)
    |
    +----> [Resize: 672px] --> VFM (frozen) --> Feature Map C (high-res)
    
    v
[Upsample all to common size]
    v
[Fusion: Average or Learned Module]
    v
Unified Multi-Scale Representation

Think of MuRF like a photographer using multiple lenses on the same scene. A wide-angle lens (low-res) captures the entire landscape—you see the forest. A telephoto lens (high-res) zooms into a single tree—you see the bark texture. Normally, you’d pick one lens and miss what the other sees. MuRF is like overlaying the photos from all lenses: the wide-angle tells you where the tree is in the scene, the telephoto tells you what kind of tree it is. The fusion layer is the darkroom where you blend the exposures. The camera (VFM) never changes—you’re just using it multiple times and combining the shots.

Key Concepts

• Inductive Bias Across Scales: At low resolution, the model is forced to compress information, which naturally emphasizes global structure and semantic categories (a car is still recognizable at 224px). At high resolution, the model has bandwidth to encode local texture, edges, and spatial precision (you can see the license plate). These aren’t redundant—they’re complementary. Low-res gives you the “what” and “where roughly”; high-res gives you the “exactly where” and “what specifically.” MuRF exploits this by letting each scale do what it does best, then combining the strengths.

• Training-Free Fusion: Most multi-scale methods require end-to-end training with scale augmentation baked in. MuRF sidesteps this by operating purely at inference. The VFM weights are frozen—no gradients, no fine-tuning. The fusion module (if learned) is tiny and trained on a small dataset, or you can just average features (zero training). This makes MuRF universally applicable: you can take a VFM trained by someone else, never touch its weights, and still unlock multi-scale reasoning. It’s a post-hoc enhancement, not a training paradigm.

• Feature Space Fusion vs. Pixel Space Fusion: You could naively fuse images at the pixel level (blend the 224px and 448px images, then feed to VFM). But that destroys information—pixels don’t align semantically across scales. MuRF fuses in feature space: each scale produces a semantic representation (feature map), then those representations are combined. This preserves the distinct inductive biases of each scale while allowing them to inform each other. It’s like merging two maps of the same city—one showing roads, one showing buildings—into a single layered map.

Framework Shift

Before (mainstream approach):        After (this paper):

Input Image                          Input Image
    |                                    |
    v                                    +---> [Multi-Scale Processing]
[Resize to fixed size]                   |     (224px, 448px, 672px)
    |                                    |
    v                                    v
VFM (single scale)                   VFM (frozen, applied 3x)
    |                                    |
    v                                    v
Feature Map                          [Feature Fusion Module]
    |                                    |
    v                                    v
Task Head                            Unified Multi-Scale Features
                                         |
                                         v
                                     Task Head

From single-scale inference to multi-scale fusion, the core shift is: stop choosing a resolution, start combining them.

Expert Assessment

Problem choice: This is a real gap, not manufactured. The field has been stuck in a local optimum: VFMs are powerful, but we’ve been using them in the dumbest way possible (one scale, take it or leave it). The problem sits at the intersection of efficiency and representation quality—both critical for deployment. It’s not groundbreaking science, but it’s a practical bottleneck that needed addressing.

Method maturity: This is 80% clever insight, 20% engineering. The insight—that frozen VFMs can be reused at multiple scales and fused—is simple but non-obvious. The execution is straightforward: no exotic architectures, no complex training recipes. That’s a strength. However, the paper doesn’t deeply explore why this works (e.g., what’s happening in the feature space geometry when you fuse scales?). It’s empirical validation without much theoretical grounding. A simpler approach might be adaptive resolution selection (pick the best scale per image), but the paper doesn’t compare against that.

Experimental integrity: Baselines are fair—they compare against single-scale inference at various resolutions, which is the honest comparison. The gains are consistent across tasks (semantic segmentation, depth estimation, image retrieval), which suggests the method isn’t overfitting to one benchmark. However, the computational cost analysis is a bit hand-wavy: they claim MuRF is “efficient” because you can cache features, but in practice, you’re running the VFM 3x per image. For real-time applications, that’s a dealbreaker. The paper would be stronger if it included latency benchmarks on actual hardware.

Writing quality: The introduction oversells the “universality” angle—yes, it works on DINOv2 and SigLIP2, but that’s two model families, not a proof of universality. The method section is clear, but the ablation studies are buried in the appendix when they should be front and center (e.g., how much does each scale contribute? What happens if you only fuse two scales?). The related work section is perfunctory—it lists prior multi-scale methods but doesn’t explain why they failed to solve this problem. Rewriting the ablation section to be more prominent would elevate the paper.

Verdict: weak accept — Solid empirical work with a simple, practical idea, but lacks depth in analysis and oversells its scope.

Takeaways

Practitioners can steal the core idea: if you have a frozen model and want better representations without retraining, process inputs at multiple scales and fuse the outputs. This isn’t limited to vision—it applies to any domain where scale matters (audio spectrograms, time-series at different granularities, text embeddings at sentence vs. paragraph level). The specific fusion mechanism (averaging vs. learned module) is less important than the principle: don’t pick a scale, combine them. Also, the “training-free” framing is a useful lens for thinking about model enhancement—what can you do at inference time that doesn’t require touching the weights? That’s a transferable design pattern.