EviScreen: Evidence-Based Disease Screening with Historical Case Retrieval

Paper: 2605.15171 Authors: Chenyu Lian, Hong-Yu Zhou, Jing Qin Categories: cs.CV, cs.AI, cs.LG

The Gap

Current medical image screening models operate in isolation—they analyze each case independently without referencing historical knowledge. This creates two problems: (1) predictions lack interpretability because there’s no transparent reasoning path showing why a decision was made, and (2) performance suffers because the model can’t leverage patterns from similar past cases. Existing approaches either use post-hoc saliency maps (which don’t explain the decision process) or fail to incorporate historical evidence systematically.

Problem: Isolated screening          Assumption: Historical cases
         + poor interpretability  ->  contain reusable evidence
                                                |
                                                v
Method: Dual knowledge banks         Evidence: Higher specificity
        + retrieval + reasoning  ->  at clinical recall levels
                                                |
                                                v
                                      Conclusion: Evidence-based
                                      reasoning improves screening

The Increment

One sentence: Before this paper, disease screening models made isolated predictions with opaque reasoning; after, they retrieve region-level evidence from historical cases and show transparent reasoning paths.

Core Mechanism

EviScreen consists of three components working in sequence. First, dual knowledge banks store historical cases—one bank holds image features, the other holds abnormality maps. When a new case arrives, the system performs contrastive retrieval: it compares the current case’s regional features against the banks to find similar historical regions, pulling both the visual features and the corresponding abnormality maps.

Second, the retrieved evidence flows into an evidence-aware reasoning module. This module doesn’t just look at the current case in isolation—it cross-references the current case’s features with the retrieved historical evidence. The reasoning happens at the region level: for each area of the current image, the module asks “what do similar regions in past cases tell us?” and weighs that evidence alongside the current observations.

Third, the abnormality maps from retrieval serve dual purposes. They provide localization interpretability (showing where abnormalities are) and guide the reasoning process. Unlike post-hoc saliency maps generated after prediction, these maps emerge naturally from the contrastive retrieval process—they’re part of the evidence itself, not an explanation tacked on afterward.

Current Case                Historical Knowledge Banks
    |                              |
    |                    +---------+---------+
    v                    |                   |
[Feature                 |  [Image Bank]    [Abnormality Bank]
Extraction]              |   (features)      (location maps)
    |                    |                   |
    +--------------------+-------------------+
                         |
                         v
                [Contrastive Retrieval]
                  (find similar regions)
                         |
            +------------+------------+
            |                         |
            v                         v
    [Retrieved Features]      [Retrieved Maps]
            |                         |
            +------------+------------+
                         |
                         v
              [Evidence-Aware Reasoning]
               (current + historical)
                         |
                         v
                  [Prediction + Maps]

Think of EviScreen as a medical resident consulting with senior doctors. When the resident sees a new X-ray, they don’t just stare at it alone—they pull up similar cases from the hospital archive. For each suspicious region, they find past cases with similar-looking areas and check: “What was the diagnosis there? Where exactly was the abnormality?” The resident then makes their diagnosis by weighing what they see now against what those historical cases revealed. The archive isn’t just a reference library—it’s structured evidence that directly shapes the reasoning. The abnormality maps are like annotations from previous radiologists, showing exactly where to look, and these annotations become part of the diagnostic reasoning rather than just post-hoc explanations.

Key Concepts

• Contrastive Retrieval: Instead of retrieving entire similar cases, the system retrieves at the region level using contrastive learning. Imagine you’re diagnosing a lung X-ray with a suspicious spot in the upper right. Traditional retrieval would find “similar chest X-rays overall.” Contrastive retrieval zooms in: it finds historical cases where the upper-right region looked similar, even if the rest of the image was different. The “contrastive” part means the system learned to pull features closer for similar regions and push them apart for dissimilar ones. This gives you targeted evidence: “In 15 past cases with upper-right regions like this, 12 were malignant nodules.” That’s far more useful than “here are some vaguely similar chest X-rays.”

• Dual Knowledge Banks: The system maintains two parallel archives—one storing visual features, one storing abnormality maps. Why two? Because you need different types of evidence. The feature bank answers “what does this region look like?” while the abnormality bank answers “where were problems found in similar cases?” When you retrieve from both simultaneously, you get complete evidence: not just “this looks like case #47” but also “in case #47, the abnormality was in this exact location.” It’s like having both the X-ray images and the radiologist’s annotations in your archive—you need both to reason effectively.

• Retrospection Interpretability: This is interpretability through evidence rather than through post-hoc explanation. Most models generate saliency maps after making a prediction—they’re reverse-engineering an explanation. EviScreen’s interpretability is forward: the retrieved historical cases and their abnormality maps are the evidence used during reasoning, not explanations generated afterward. When the model says “abnormality detected,” it can point to specific historical cases and say “because these similar regions in past cases were abnormal.” The reasoning path is transparent because the evidence is explicit, not because we’re trying to visualize what a black box learned.

Framework Shift

Before (mainstream approach):        After (this paper):

  New Case                             New Case
     |                                    |
     v                                    v
[CNN Feature                         [Feature
 Extraction]                          Extraction]
     |                                    |
     v                                    +---> [Contrastive
[Classification                                 Retrieval]
 Head]                                           |
     |                                           v
     v                                    [Historical
[Prediction]                              Evidence]
     |                                           |
     v                                           v
[Post-hoc                                [Evidence-Aware
 Saliency Map]                            Reasoning]
                                                 |
Isolated reasoning,                              v
explanation after fact                    [Prediction +
                                           Evidence Maps]

                                          Evidence-based reasoning,
                                          transparent from start

From isolated prediction to evidence-based reasoning, the core shift is making historical knowledge an explicit part of the decision process rather than an implicit pattern buried in weights.

Expert Assessment

Problem choice: Real gap. Medical screening genuinely suffers from lack of interpretability and inability to reference past cases systematically. This isn’t manufactured—radiologists actually do consult historical cases, and current models don’t capture that workflow. The problem sits at the intersection of interpretability and performance, which is where the field needs to move.

Method maturity: Solid engineering with one clever insight—using contrastive retrieval to get abnormality maps as a byproduct rather than training a separate localization module. The dual knowledge bank structure is straightforward but effective. However, the paper doesn’t deeply explore failure modes: what happens when retrieved evidence is misleading? How does the system handle distribution shift between historical and current cases? These are solvable but not addressed.

Experimental integrity: Benchmarks are carefully constructed for real-world screening scenarios, emphasizing specificity at clinical recall levels (which matters more than raw accuracy). Baselines include recent interpretable methods, not just vanilla CNNs. One concern: the paper doesn’t show what happens when the knowledge banks are small or when no good matches exist. Performance likely degrades gracefully, but we don’t see that analysis.

Writing quality: The abstract and introduction are clear. The method section gets dense—Figure 2 (presumably the architecture diagram) carries too much load without enough textual walkthrough. The ablation studies are thorough but buried. If the authors rewrote Section 3.2 to lead with intuition before diving into technical details, the paper would be significantly more accessible.

Verdict: weak accept — Solid contribution with real clinical relevance, but the novelty is more in the integration than in individual components. The evidence-based reasoning framework is valuable, though not groundbreaking. Experimental validation is strong within its scope but could be more comprehensive.

Takeaways

Retrieval as reasoning, not just augmentation: Most retrieval-augmented systems treat retrieved content as context to condition on. EviScreen makes retrieval part of the reasoning mechanism itself—the retrieved abnormality maps directly shape the prediction. This pattern transfers: in any domain where decisions should reference historical precedents (legal case analysis, financial fraud detection), make the retrieval evidence explicit in the reasoning path rather than just feeding it as additional input.

Dual-bank architecture for multi-modal evidence: Storing features and annotations separately, then retrieving from both, is a clean pattern for any task where you need both “what does this look like?” and “what was the outcome?” evidence. In code review, you’d store code embeddings in one bank and bug reports in another. In customer support, query embeddings in one bank and resolution notes in another.

Contrastive retrieval for localization: Using contrastive learning to retrieve region-level matches, then deriving localization maps from the retrieval process itself, is cheaper than training separate localization modules. This works when your task naturally involves comparing parts of inputs (medical images, documents, time series). The localization emerges from similarity rather than being explicitly supervised.