Human-level 3D Shape Perception Emerges from Multi-view Learning

Paper: 2602.17650 Authors: Tyler Bonnen, Jitendra Malik, Angjoo Kanazawa Categories: cs.CV

Abstract

This paper presents a breakthrough in computational 3D perception by developing neural networks that match human-level performance on 3D shape inference tasks. The key innovation lies in training models with a visual-spatial objective over naturalistic multi-view data—learning to predict camera locations and visual depth from images taken at different positions within natural scenes. Remarkably, these models achieve human-level accuracy on established 3D perception benchmarks without any task-specific training or fine-tuning. The framework not only matches human accuracy but also predicts fine-grained behavioral patterns including error distributions and reaction times, suggesting a fundamental correspondence between the model’s learned representations and human perceptual mechanisms.

Key Contributions

• First computational model to achieve human-level accuracy on 3D shape inference tasks without task-specific training
• Novel multi-view learning framework that learns from visual-spatial objectives over naturalistic sensory data
• Zero-shot evaluation methodology demonstrating generalization to arbitrary 3D perception tasks
• Behavioral correspondence analysis showing models predict human error patterns and reaction times
• Elimination of object-specific inductive biases, relying instead on general visual-spatial learning principles

Methodology and Architecture

The research introduces a paradigm shift in how we approach 3D perception modeling. Traditional methods have relied heavily on object-specific priors, geometric constraints, or supervised learning with 3D annotations. This work takes a fundamentally different approach inspired by the sensory information available to biological vision systems.

The multi-view learning framework operates on a simple principle: given multiple images from different viewpoints within a natural scene, the model learns to predict spatial relationships between these views. Specifically, the training objective involves predicting camera pose (location and orientation) and depth information without explicit 3D shape supervision. This visual-spatial objective mirrors the kind of information humans naturally extract from visual experience as they move through environments.

The neural architecture processes sets of images and learns representations that encode viewpoint-invariant shape information as an emergent property. Critically, the models contain no explicit 3D shape representations, object categories, or geometric primitives—all structure emerges from the multi-view consistency constraints implicit in the training data.

Experimental Results and Human Comparison

The evaluation framework centers on a well-established psychophysical paradigm for measuring 3D shape perception. Human participants and models are presented with identical visual stimuli and must make judgments about 3D object structure. The zero-shot nature of the evaluation is crucial: models are tested on this task without any exposure to the specific stimuli or task format during training.

The results demonstrate that multi-view trained models achieve accuracy statistically indistinguishable from human performance. This represents the first time a computational model has reached human-level performance on this benchmark without task-specific optimization.

Beyond aggregate accuracy, the analysis reveals striking correspondences in error patterns. When humans make mistakes on particular stimuli, the models tend to make similar errors. This correlation extends to reaction time predictions—model processing dynamics (measured through iterative inference or layer depth) correlate with human response latencies. These behavioral signatures suggest the models have discovered computational strategies similar to those employed by human visual systems.

Implications for Vision Science and AI

The success of this approach carries profound implications for both neuroscience and artificial intelligence. From a scientific perspective, the results suggest that human-level 3D perception may not require the elaborate geometric reasoning, object knowledge, or physical intuition often hypothesized as necessary. Instead, 3D shape understanding can emerge from a relatively simple learning objective applied to naturalistic visual-spatial data.

This finding aligns with ecological theories of perception emphasizing the information available in natural viewing conditions. As humans move through environments, they naturally experience multi-view visual information. The correspondence between model and human behavior suggests that human 3D perception may be fundamentally grounded in learning from such multi-view experiences.

For AI systems, the work demonstrates a scalable path toward human-level perceptual capabilities. The training approach requires only multi-view image data—no manual 3D annotations, object labels, or hand-crafted geometric constraints. This makes the framework applicable to diverse visual domains and potentially extensible to other perceptual modalities.

The behavioral correspondence between models and humans also opens new possibilities for using these models as computational hypotheses in vision science. The models can generate predictions about human perception in novel conditions, potentially guiding experimental design and theory development.

Takeaways

1. Human-level 3D shape perception can emerge from multi-view learning over naturalistic visual-spatial data, without object-specific inductive biases or task-specific training.

2. Models trained with visual-spatial objectives (predicting camera pose and depth from multi-view images) achieve zero-shot performance matching human accuracy on established 3D perception benchmarks.

3. The correspondence extends beyond accuracy to fine-grained behavioral patterns—models predict human error distributions and reaction times, suggesting shared computational principles.

4. This approach eliminates the need for 3D annotations, geometric constraints, or object category labels, offering a scalable framework for perceptual AI systems.

5. The findings support ecological theories of perception, suggesting human 3D vision may be fundamentally grounded in learning from natural multi-view visual experience rather than explicit geometric reasoning.