BenchX: Benchmarking AI Models for Cancer Detection and Localization with Demographic and Protocol Biases

Paper: 2606.24883
Authors: Qi Chen, Wenxuan Li, Pedro R. A. S. Bassi, Xinze Zhou, Jakob Wasserthal, Ibrahim Ethem Hamamci, Sezgin Er, Ashwin Kumar, Yiwen Ye, Yuhan Wang
Categories: cs.CV

The Gap

Existing medical AI benchmarks evaluate models on average accuracy over large, mixed cohorts. For cancer detection, the field has assumed that a model trained on diverse data will generalize uniformly across patient subgroups (age, sex, race) and imaging protocols (contrast phase, slice thickness). Prior work like MedMNIST or the RSNA challenge datasets either control only one variable (e.g., imaging modality) or lack subgroup labels entirely. The result: a model may score 95% AUC on the whole test set but drop to 60% on young African American women — and no one knows because no one tests at that granularity.

BenchX fills this gap by:

1. Constructing a massive (85,355 CT scans) multi-site, multi-protocol dataset.
2. Using LLMs to automatically extract demographic and protocol metadata from free-text clinical reports.
3. Stratifying evaluation by tumor size, location, patient subgroup, and imaging protocol.
4. Showing that 12 state-of-the-art models all suffer systematic performance degradation on rare subgroups.

[ASCII logic topology: Problem -> Assumption -> Method -> Evidence -> Conclusion]

  Problem: Models fail on rare subgroups, but no benchmark detects this.
     |
     v
  Assumption: Average accuracy hides subgroup disparities.
     |
     v
  Method: Build large multi-site dataset, use LLM to extract subgroup labels,
          then stratify evaluation by subgroup.
     |
     v
  Evidence: 12 SOTA models drop 15-30% AUC on young female African Americans vs average.
     |
     v
  Conclusion: Subgroup-level evaluation is essential for reliable clinical AI.

The Increment

One sentence: Before BenchX, we knew medical AI could be biased, but we had no systematic tool to measure exactly where and how much for tumor detection — after BenchX, we have a public, reproducible benchmark that exposes the failure patterns of 12 models across 85k+ scans with demographic and protocol granularity.

Core Mechanism

The benchmark pipeline has three main components:

1. Data aggregation: 85,355 CT scans are pooled from multiple hospitals and public sources (TCIA, LiTS, etc.), covering various cancer types (liver, lung, kidney, pancreas). Each scan comes with its original clinical report (free text) and segmentation masks for tumors.

2. LLM-based metadata extraction: Instead of relying on manually annotated demographic fields (often missing or inconsistent), the authors feed each clinical report to an LLM (GPT-4 or similar) with a structured prompt asking: “What is the patient’s age, sex, race? What contrast phase was used? What is the slice thickness?” The LLM outputs structured JSON fields. Human validation on a 1,000-sample subset shows >95% accuracy.

3. Stratified evaluation: Twelve published tumor detection models (nnUNet, Swin UNETR, CoTr, etc.) are run on the full dataset. Results are then broken down by tumor size (<10mm, 10-30mm, >30mm), location (liver, lung, etc.), patient subgroup (age bins, sex, race combinations), and imaging protocol (non-contrast, arterial, portal venous, etc.). Metrics include detection sensitivity, false positives per scan, and localization accuracy (Dice score on detected tumors).

Data flow: CT scans + reports -> LLM extracts metadata -> model predictions -> stratify by metadata -> report per-subgroup performance.

[ASCII diagram of method internals]

                       +-----------------------+
                       | Clinical Reports (txt) |
                       +-----------------------+
                              |
                              v
                     +-------------------+
                     | LLM (GPT-4)       |
                     | "extract: age,    |
                     |  sex, race, phase, |
                     |  slice thickness"  |
                     +-------------------+
                              |
                              v
                    +------------------+
                    | Structured JSON  |
                    | metadata per scan|
                    +------------------+
                              |
          +-------------------+-------------------+
          |                                       |
          v                                       v
+----------------------+           +----------------------+
| CT Scan Volume (nii) |           | Tumor Masks (nii)    |
+----------------------+           +----------------------+
          |                                       |
          v                                       v
+----------------------+           +----------------------+
| 12 Detection Models  |           | Evaluation Scripts   |
| (nnUNet, Swin UNETR, |           | (sensitivity, FP,    |
|  CoTr, ...)          |           |  Dice)               |
+----------------------+           +----------------------+
          |                                       |
          +---------> combine results <-----------+
                              |
                              v
                   +----------------------------+
                   | Stratified Report by       |
                   |   - tumor size             |
                   |   - location               |
                   |   - patient subgroup       |
                   |   - imaging protocol       |
                   +----------------------------+

Now let me give you a structural metaphor: think of BenchX as a hospital quality audit team that visits a dozen radiology AI systems.

• CT scans are the “patients” coming through the door.
• The LLM is a hyper-efficient nurse who reads every patient’s intake form and stamps them with colored wristbands: blue for age>60, orange for contrast phase A, etc.
• The 12 detection models are the “diagnostic robots” being audited.
• The stratified evaluation is the audit committee’s final report, which doesn’t just give an average “pass rate” but breaks it down: “Robot A does great on blue-band patients but terrible on orange-band ones, especially if the wristband also says ‘young female’.”

Without the wristband (metadata), the hospital only sees overall accuracy and misses dangerous blind spots. BenchX provides the wristbands at scale.

Key Concepts

• Subgroup stratification: Instead of lumping all test samples together, you split them by attributes like age, sex, race, or imaging protocol. Example: measure model sensitivity separately for “female, aged 20-40, African American” vs “male, aged 60-80, Caucasian”. If the model performs well on the latter but poorly on the former, that’s a signal of demographic bias — not just random variation. This is the core analytic that BenchX makes possible.

• LLM-based metadata extraction: Clinical reports are unstructured free text that often contain patient demographics and imaging parameters but buried in narrative. Manually labeling 85k scans is impractical. BenchX uses a large language model (GPT-4) with a few-shot prompt to extract structured fields. Think of it as a super-powered regex — it can handle “Patient is a 32-year-old AA female” or “32 yo African American woman” and output a clean JSON: age=32, sex=F, race=Black. This makes the benchmark scalable and reproducible.

• Distribution shift & robustness: A model trained on one hospital’s data (mostly older patients, one contrast phase) may fail when deployed at another site with different demographics or protocols. BenchX quantifies this by reporting performance across protocol-specific subsets. For example, a model might have 90% sensitivity on portal-venous phase scans but only 60% on non-contrast scans. This tells practitioners: “Don’t trust this model on non-contrast cases.”

Framework Shift

Before (mainstream approach):        After (this paper):
+-----------------------------+      +-------------------------------+
| Train on mixed dataset     |      | Train on mixed dataset        |
| Test on mixed dataset      |      | Test on mixed dataset         |
| Report average AUC         |      | Report AUC by subgroup        |
| (single number)            |      | (heatmap of 200 subgroups)    |
+-----------------------------+      +-------------------------------+
       |                                    |
       v                                    v
  "Model works at 0.95 AUC"           "Model works at 0.95 AVG,
  Blind to subgroup failures           but 0.60 on young AA women,
                                       0.70 on non-contrast scans"
       |                                    |
       v                                    v
  Deploy and get surprised            Anticipate and mitigate
  by real-world failures               before deployment

One sentence: From average-centric evaluation to subgroup-by-subgroup auditing — the core shift is treating population heterogeneity as first-class measurement, not noise.

Expert Assessment

Problem choice: Real gap. The medical AI community has paid lip service to fairness but rarely built tools to measure it at scale in tumor detection. This paper fills a genuine practical need. It sits squarely at the intersection of medical imaging, fairness, and LLM application.

Method maturity: Clever, not brute force. The use of LLMs to extract metadata is a pragmatic insight that avoids the prohibitive cost of manual labeling. However, the reliance on GPT-4 introduces a variable-cost dependency and possible LLM biases (e.g., poor performance on non-English reports if the dataset is multilingual). A simpler alternative could be rule-based extraction, but the authors show LLM outperforms regex on their validation.

Experimental integrity: Baselines are fair — they test all 12 models on the same data. The main concern is that the LLM extraction accuracy (claimed >95%) was validated on only 1,000 samples; a larger independent validation would strengthen confidence. Also, the paper does not report whether the LLM’s extraction errors correlate with certain subgroups (e.g., does GPT-4 mis-extract race more often for Hispanic patients?), which could introduce systematic bias in the benchmark itself.

Writing quality: The abstract is clear and the figures (in the full paper) are effective. However, the methods section in the arXiv preprint is dense and skips some implementation details (e.g., exact prompt template for LLM, handling of missing metadata). A rewritten methods section with a concrete example of the LLM input/output would significantly improve reproducibility.

Verdict: Strong accept — provides an urgently needed benchmark, introduces a scalable metadata extraction method, and exposes concrete failure modes that should change how medical AI is evaluated.

Takeaways

• Practitioner steal: If you’re building a medical AI model, replicate the stratified evaluation approach: after training, break your validation set by age, sex, race, and imaging protocol. BenchmarkX’s code provides the machinery; you just need your own metadata.
• LLM trick: Use an LLM to extract structured metadata from free-text clinical reports. This is transferable to any medical dataset where structured labels are missing. Write a clear few-shot prompt and validate on a held-out set — the paper’s validation protocol is a template.
• New metric: Report not just average performance but a “worst-case subgroup performance” or “subgroup gap” (max difference between any subgroup and the average). This becomes a diagnostic tool for model robustness.
• Dataset: The 85k-scan dataset itself, with extracted metadata, will be released. Any researcher can now benchmark their own model against these 12 without collecting new data — a huge time saver.