CoFiDA-M: Baking Clinical Reasoning into Image-Only Skin Cancer Screening

Paper: 2605.31591 Authors: Nurjahan Sultana, Moi Hoon Yap, Xinqi Fan, Wenqi Lu Categories: cs.CV

The Gap

Domain adaptation for medical imaging has focused on aligning visual distributions between source (expert dermoscopic) and target (consumer clinical) domains. Methods like adversarial alignment or self-training treat images as pixel patterns, ignoring that dermatologists reason through clinical concepts—asymmetry, border irregularity, color variation. Foundation models like MONET can now extract these concepts as dense probabilistic maps, but here’s the paradox: this metadata exists only at training time. At deployment, a real-world screening app receives just a smartphone photo. Existing methods can’t leverage concepts during training without requiring them at test time, leaving a deployment gap between what helps learning and what’s available in practice.

Problem: Expert images → Consumer images (distribution shift)
   |
   v
Prior approaches: Align pixel distributions
   |                (ignore semantic invariants)
   v
Limitation: Can't use concept metadata at test time
   |           (deployment paradox)
   v
This paper: Train with concepts, deploy image-only
   |           (privileged information distillation)
   v
Evidence: +5.8% melanoma recall vs. state-of-the-art
   |
   v
Conclusion: Semantic reasoning can be "baked" into weights

The Increment

One sentence: Before, domain adaptation either ignored clinical concepts or required them at test time; after, models learn from concepts during training but deploy as pure image classifiers.

Core Mechanism

CoFiDA-M operates in two phases. First, a teacher network receives both images and MONET concept probability maps (dense spatial grids indicating likelihood of clinical features like asymmetry or irregular borders). The teacher uses FiLM (Feature-wise Linear Modulation) layers to transform visual features based on concept probabilities—essentially “editing” the feature space to emphasize semantically relevant patterns. This produces a concept-aware representation that bridges domain gaps through clinical invariants rather than pixel statistics.

Second, a lightweight student network receives only images. Instead of mimicking the teacher’s final predictions (standard knowledge distillation), the student is trained to reproduce the teacher’s edited feature representation. This forces the student to internalize the semantic transformations the teacher performed with explicit concept guidance. The student’s weights effectively encode “if I had concept maps, I would adjust features this way”—the clinical reasoning becomes implicit.

At deployment, the student runs standalone on consumer images, no metadata required. The concept-guided feature editing that required MONET at training time is now embedded in the student’s learned transformations.

Training Phase:
Image + Concepts ──> Teacher (FiLM modulator) ──> Edited Features
                                                         |
                                                         v
Image only ──────────> Student ──────────────> Mimic Edited Features
                                                         |
                                                         v
                                                   Final Prediction

Deployment Phase:
Consumer Image ──> Student (concepts baked in) ──> Prediction

Think of it like learning to cook from a master chef who uses precise thermometers and pH meters. The teacher (chef) adjusts seasoning based on exact measurements (concept probabilities). The student (apprentice) watches not just the final dish but how the chef’s hand moves when adding salt after seeing a pH reading. Through repeated observation, the apprentice learns to estimate “this needs more acid” from visual cues alone—the thermometer’s guidance is now internalized as intuition. At deployment, the apprentice cooks without instruments, but the precision training shaped their instincts. The FiLM modulator is the chef’s hand adjusting features; the student learns to make those adjustments from visual cues that correlate with concepts.

Key Concepts

• Privileged Information: During training, you have extra data (here, concept maps from MONET) that won’t be available at test time. Instead of ignoring it or requiring it forever, you use it to guide learning, then distill that guidance into a model that doesn’t need it. It’s like training a radiologist with textbook annotations overlaid on X-rays, then deploying them to read unmarked films—the annotations shaped their perception but aren’t needed anymore. The key insight: metadata can teach without being a runtime dependency.

• FiLM (Feature-wise Linear Modulation): A mechanism to conditionally transform neural network features based on external information. Given a feature map and a conditioning signal (here, concept probabilities), FiLM applies learned affine transformations: output = gamma ** features + beta, where gamma and beta are predicted from the conditioning signal. It’s like a volume knob and tone control for each feature channel—concept probabilities tell the network “turn up asymmetry-sensitive features, turn down texture noise.” This lets the network dynamically re-weight what it pays attention to based on semantic context.

• Feature-Space Distillation: Traditional distillation matches a student’s output probabilities to a teacher’s. Feature-space distillation matches intermediate representations. Why? Output probabilities collapse all reasoning into a single vector; intermediate features preserve the structure of how the teacher “thinks.” By matching edited features rather than final predictions, the student learns the transformation process itself—not just the answer, but the reasoning path. It’s the difference between copying someone’s essay versus learning their outlining method.

Framework Shift

Before (mainstream approach):        After (this paper):

Source ──> Align ──> Target          Source + Concepts ──> Teacher
 (pixels)   (adversarial/           (semantic editing)      |
            self-training)                                   v
                                                      Edited Features
                                                             |
                                                             v
                                     Target (image only) ──> Student
                                                      (mimics editing)

Focus: Pixel distribution            Focus: Semantic transformation
Runtime: Image only                  Runtime: Image only
Training: Image only                 Training: Image + metadata

From aligning what images look like to aligning how models reason about them, the core shift is treating concepts as training-time scaffolding rather than runtime requirements.

Expert Assessment

Problem choice: Real and well-motivated. The deployment paradox—foundation models provide rich metadata but practical apps can’t assume it—is a genuine barrier to clinical adoption. The problem sits at the intersection of domain adaptation and practical deployment, addressing a gap that matters beyond dermatology (any field where expert annotations exist at training time but not inference).

Method maturity: Clever use of existing components (FiLM, feature distillation) rather than inventing new architectures. The insight is in the training regime, not the modules. However, the reliance on MONET as the concept source is a potential brittleness—MONET’s own errors propagate. A simpler baseline would be training the student directly on source+target with standard augmentation; the paper doesn’t deeply explore whether the concept detour is necessary or just helpful.

Experimental integrity: Baselines are reasonable (DANN, CDAN, self-training), and the multi-dataset evaluation (Derm7pt, ISIC, PAD-UFES) is stronger than single-dataset claims. Melanoma recall improvements (+5.8% over best baseline) are clinically meaningful. However, the paper doesn’t ablate the cost of the two-stage training—could a single end-to-end model with concept auxiliary losses achieve similar results? The comparison focuses on final performance, not training efficiency or model size.

Writing quality: The abstract and introduction are clear, but the method section buries the key insight (feature-space distillation vs. output distillation) in dense notation. Figure 1 tries to show the full pipeline but ends up cluttered. Rewriting Section 3.2 to lead with the cooking metaphor, then formalize, would make the contribution land faster. The related work section is thorough but could trim 30% without losing substance.

Verdict: Weak accept—the problem is real, the solution is practical, and the results are solid. The method isn’t groundbreaking but it’s a smart engineering contribution that others can build on. The writing needs tightening, and the ablations could be deeper, but the core idea holds up.

Takeaways

Privileged information as training scaffolding: If you have rich metadata at training time (annotations, auxiliary sensors, foundation model outputs) but not at test time, don’t discard it—use it to guide feature learning, then distill into a metadata-free model. This pattern transfers to any domain with training-time-only signals: video models trained with optical flow but deployed on frames alone, NLP models trained with parse trees but deployed on raw text.

Feature distillation over output distillation: When the teacher’s value is in how it processes information (not just what it predicts), match intermediate representations. This preserves the reasoning structure. Useful when the teacher has privileged information or architectural advantages the student can’t replicate.

FiLM for conditional feature editing: If you need to modulate a network’s behavior based on external signals (metadata, context, user preferences), FiLM layers are a lightweight, interpretable mechanism. They’re more parameter-efficient than concatenating signals and re-learning everything.

Concept-based domain adaptation: When domains differ in low-level statistics but share high-level semantics (expert vs. consumer medical images, synthetic vs. real robotics data), aligning through semantic invariants (concepts, object parts, scene structure) can be more robust than pixel-level alignment. Look for foundation models or auxiliary tasks that extract these invariants.