Spherical-to-ERP Epipolar Rectification for Single-Axis Disparity in 360 Stereo

Paper: 2606.24847 Authors: Sahereh Obeidavi, Dieter Landes Categories: cs.CV

The Gap

Classical disparity estimation relies on rectilinear images where epipolar lines are horizontal (or vertical), reducing the search to a single axis. For omnidirectional stereo (spherical or fisheye views), the epipolar curves follow great circles on the sphere, producing two-dimensional pixel displacements. Prior work either designed custom spherical stereo algorithms (e.g., spherical CNN features, spherical optical flow) or performed complex rectification that warps the sphere to a rectilinear grid with high distortion. These approaches are often slow, require retraining, or lose consistency.

This paper asks a pragmatic question: Can we simply use equirectangular (ERP) projection — a common 360° representation — as a pre-processing step to straighten epipolar curves, then feed the result into an off-the-shelf stereo pipeline (RAFT+EACS) and get good disparity? They answer yes, with synthetic experiments confirming real-time, accurate, and smooth disparity maps.

                Problem                           Assumption
                   v                                 v
   Spherical stereo -> 2D displacement       ERP straightens
   (curved epipolars)    hard to estimate     epipolar curves?
                   |                                 |
                   v                                 v
                Method:  Spherical -> ERP -> RAFT+EACS
                   |                                 |
                   v                                 v
              Evidence: Synthetic fisheye      Conclusion: ERP
              datasets show accurate, smooth   preproc enables
              disparity maps at real-time speed standard pipeline

The Increment

One sentence: Before this paper, spherical stereo required special algorithms or heavy rectification; after, it can be handled by a simple ERP reprojection followed by a conventional stereo pipeline (RAFT+EACS) — no retraining needed.

Core Mechanism

The pipeline has four stages:

1. Spherical input: Two fisheye/spherical cameras (left-right or top-bottom) capture overlapping 360° views.
2. ERP projection: Each spherical image is projected onto an equirectangular grid (longitude on x-axis, latitude on y-axis). This step bends the epipolar curves — originally great circles on the sphere — into nearly straight horizontal lines (for a horizontal baseline) or vertical lines (for a vertical baseline).
3. RAFT dense optical flow: The two ERP images are fed into RAFT, a state-of-the-art recurrent optical flow network. It computes a full 2D flow field between the two views.
4. EACS channel selection: The Epipolar-Aligned Channel Selection (EACS) module, introduced in the authors’ prior work, isolates the component of the flow that is aligned with the stereo baseline. For a horizontal rig, it keeps only the horizontal flow; for a vertical rig, only the vertical flow. This reduces the 2D flow to a 1D disparity map.

   Left ERP image         Right ERP image
          |                     |
          +----------+----------+
                     |
                  v v v
                RAFT -> [2D flow]
                     |
                  EACS -> [1D disparity] 
                     |
              (horizontal or vertical)

Structural metaphor: Think of the pipeline as a map-making factory.

• The spherical image is like a globe with curved lines of longitude and latitude (epipolar curves are great circles, like the prime meridian).
• The ERP projection is like unfolding the globe into a flat world map (like the Mercator projection). On that flat map, lines of latitude become straight horizontal lines, and lines of longitude become straight vertical lines.
• RAFT is a worker who measures the shift in position of every point between the left and right maps. On a flat map, the shift is mostly horizontal (if the two maps are from left and right cameras) — but there can be slight vertical jitter due to lens distortion or misalignment.
• EACS is the quality inspector who throws away the vertical jitter and keeps only the horizontal shift. That horizontal shift is exactly what we call disparity.

Without the ERP step, the worker would see curved shifts — hard to interpret. With ERP, the world is flattened, the shifting is almost pure horizontal, and the inspector’s job becomes trivial.

Key Concepts

• Epipolar geometry: Given two camera views, a point in the left image must lie along a line (the epipolar line) in the right image. For rectilinear cameras, these lines are horizontal; for spherical cameras, they are great circles. ERP projection makes these great circles into straight lines, so the search reduces to one dimension.

• Equirectangular (ERP) projection: A cylindrical projection mapping longitude (0°–360°) to the x-axis and latitude (-90°–90°) to the y-axis. It preserves lines of latitude as straight, at the cost of severe distortion near the poles. For stereo, the key property is that great circles (epipolar lines) become nearly straight curves aligned with the x- or y-axis, depending on the baseline direction.

• RAFT+EACS: RAFT (Recurrent All-Pairs Field Transforms) is a high-performance optical flow network. EACS is a simple post-processing step that selects only the flow component parallel to the baseline (e.g., horizontal for left-right stereo). The combination, originally designed for rectilinear and ERP stereo, works here because the ERP pre-processing enforces the epipolar alignment.

Framework Shift

Before (mainstream approach):                After (this paper):

Spherical image pairs                        Spherical image pairs
        |                                             |
   Custom spherical stereo                   ERP projection (preprocessing)
   (e.g., spherical CNN,                         |
    spherical flow)                         Rectified ERP images
        |                                             |
   2D disparity search                        RAFT+EACS (standard)
   (complex, slower)                               |
                                            1D disparity map (smooth)

One sentence: From curved epipolar geometry requiring 2D matching to flattened epipolar geometry enabling 1D matching via a simple reprojection, the core shift is that ERP projection makes the problem geometrically compatible with conventional stereo pipelines.

Expert Assessment

Problem choice: Real gap. 360° stereo is increasingly used in VR, robotics, and autonomous driving. Prior approaches either required custom networks or suffered from distortion. The idea of using ERP as a generic rectification step is straightforward and practical — the gap is whether a generic pipeline can match performance of specialized methods. The paper closes that gap convincingly for synthetic data.

Method maturity: Clever insight with brute-force validation. The core insight — ERP straightens epipolar curves — is well-known in the community. What’s new is combining it with RAFT+EACS and showing it works. The method is not reframing the field, but it provides a solid engineering baseline. Simpler alternatives? Possibly using optical flow directly on sphere with great-circle constraints, but that would be more complex.

Experimental integrity: Synthetic datasets only. This is the biggest weakness. Real-world 360° cameras have lens distortion, rolling shutter, and calibration errors. The results on synthetic data might not transfer. Baselines are fair: they compare to a standard RAFT+EACS on ERP images (which is exactly their method) and likely to a naive RAFT on original fisheye? The paper mentions “synthetic fisheye stereo datasets” but doesn’t name specific benchmarks. No comparison to spherical-specific methods (e.g., OmniFlow). Red flag: only one dataset, synthetic. Metrics: accuracy, smoothness, consistency — but no absolute error against ground truth.

Writing quality: Adequate but dry. The abstract is clear, but the method section could use a diagram of the spherical-to-ERP transformation. The biggest cut corner: no real-world experiments. If they had tested on, say, the 360SD-Net dataset, the paper would be much stronger. The result section is a bit thin.

Verdict: Weak accept — valuable engineering contribution but limited by synthetic-only evaluation and lack of comparison to spherical-specific methods.

Takeaways

• ERP as rectifier: If you ever need to run standard stereo or optical flow on spherical images, reproject to ERP first. It works surprisingly well for horizontal/vertical baselines.
• RAFT+EACS as off-the-shelf tool: The combination (dense flow + channel selection) is a modular way to get disparity from any stereo pair after rectification. Steal the EACS idea: it’s just a projection of the flow onto the baseline direction.
• Synthetic validation is not enough: Practitioners should test on real 360° cameras before deploying this pipeline. The paper’s real takeaway is that the *approach is sound, but real-world results may need additional calibration or denoising.