ConceptSMILE: An Independent Auditor for Concept-Based AI Explanations

Paper: 2607.09649 Authors: Mohadeseh Mollapour, Koorosh Aslansefat, Zeinab Dehghani, Bhupesh Kumar Mishra, Tejal Shah, Zhibao Mian Categories: cs.AI

The Gap

Concept-based explainable AI (think TCAV, Concept Bottleneck Models, or the wave of methods that decompose neural network reasoning into human-understandable concepts like “vessel tortuosity” or “optic disc pallor”) has made great strides in making models legible. You can now ask a medical imaging model “why did you flag this retina?” and get back something like “because of concept X with confidence 0.8.” That’s a huge improvement over opaque saliency maps.

But here’s the problem nobody’s really solved: just because a model says its reasoning involves concept X doesn’t mean concept X is actually driving the decision. Concept-level explanations can be plausible-sounding nonsense. SMILE (the predecessor) audits feature-level attributions by perturbing inputs and checking whether attribution maps behave consistently. It’s a solid idea. But it operates at the pixel/region level — it tells you whether your heatmap is reliable, not whether the *concept it claims to detect is the real causal story.

So the gap is: we have increasingly sophisticated concept-based XAI methods, but no model-agnostic, perturbation-based way to independently audit concept-level reliability. This paper fills that gap.

Concept-based XAI methods explain via
human-understandable concepts
    |
    v
But: concept outputs are treated as
ground truth -- no independent audit
    |
    v
Prior auditing (SMILE) works at
feature/pixel level, not concept level
    |
    v
[This paper: ConceptSMILE]
Extend perturbation auditing to concepts
    |
    v
Perturb regions --> measure concept shifts
--> fit surrogate --> score reliability
    |
    v
Result: concept reliability varies across
concepts and pathways (MedSAM vs VLM)

The Increment

One sentence: Before this paper, you could generate concept-based explanations and hope they were faithful; after this paper, you have a model-agnostic auditing framework that stress-tests concept explanations by perturbing inputs and measuring whether concept responses behave consistently.

Core Mechanism

ConceptSMILE works in five stages, flowing from raw input perturbation to a multi-dimensional reliability score.

Stage 1: Perturbation. Given an input image, the framework identifies the input regions that correspond to detected concepts (e.g., retinal vessels, optic disc). It then applies systematic perturbations to these regions — masking, noise injection, or occlusion — to create modified versions of the image.

Stage 2: Concept-Response Measurement. For each perturbed input, the framework runs the concept extraction pipeline (e.g., MedSAM segmentation or VLM semantic extraction) and records how each concept’s activation score shifts. This produces a dataset of (perturbation, concept-shift) pairs.

Stage 3: Locality Weighting. Not all perturbations are equally informative. The framework applies locality weighting — perturbations closer to the concept’s spatial region of interest get higher weight in the analysis. This ensures the audit focuses on whether the concept is locally sensitive to relevant inputs, not just globally correlated.

Stage 4: Surrogate Fitting. An XGBoost model is fitted on the weighted perturbation-shift data to approximate the local concept behavior. This surrogate acts as a simplified, interpretable model of how the concept extraction actually responds to input changes.

Stage 5: Reliability Scoring. The surrogate and original concept outputs are evaluated across five dimensions: attribution accuracy (does the concept attribution match ground truth?), surrogate fidelity (how well does the XGBoost approximate real behavior?), faithfulness (does changing the concept-relevant region actually change the concept?), stability (is the concept output consistent under minor perturbations?), and consistency (do related concepts behave coherently?).

Input Image
    |
    v
[Region Identification]
    |              \
    v               v
[Perturbation]    [Original Concept Extraction]
    |                        |
    v                        v
[Perturbed Concept Extraction]
    |
    v
[Concept-Shift Measurement]
    |
    v
[Locality Weighting]
    |
    v
[XGBoost Surrogate Fitting]
    |
    v
[Reliability Scoring across 5 dimensions]
    |
    v
Report: attribution accuracy, fidelity,
faithfulness, stability, consistency

The Structural Metaphor: The Insurance Auditor

Imagine you’re an insurance company reviewing a client’s financial disclosures. The client (the XAI method) tells you “I’m solvent because of these five assets” — that’s the concept-based explanation.

ConceptSMILE is the auditor who doesn’t just read the disclosure. The auditor pokes at the books. They freeze one asset account (perturbation), watch whether the reported solvency shifts (concept-response measurement), and weight their attention toward assets that are actually material to the bottom line (locality weighting). They then build a simplified model (the XGBoost surrogate) of how the client’s reported financial health actually responds to changes in individual accounts. Finally, they score the disclosure on multiple dimensions: Are the reported asset values accurate? Does the simplified model match the real behavior? If you remove a claimed asset, does the picture actually change? Is the disclosure stable if you ask the same question twice? And do related accounts tell a consistent story?

The key insight is the same in both domains: you don’t trust disclosures at face value — you stress-test them with perturbations and measure whether the stated story holds up under scrutiny.

Key Concepts

• Concept Attribution Accuracy: Think of this as “does the spotlight actually shine where the story says it shines?” When a model says “I detected vessel tortuosity in this region of the retina,” attribution accuracy asks: if you look at what the model actually responds to spatially, does it match that region? It’s like a witness pointing to a suspect in a lineup — the question isn’t just whether they point, but whether they’re pointing at the right person for the right reason. ConceptSMILE measures this by comparing the spatial sensitivity of concept extraction against ground-truth segmentations.

• Surrogate Fidelity (R-squared): When a concept extraction method is complex (like a large segmentation model), it’s hard to reason about its behavior directly. So ConceptSMICE fits a simpler model — an XGBoost — on perturbation data to approximate how the real method behaves. Surrogate fidelity measures how good that approximation is. An R-squared of 0.85 means the XGBoost captures 85% of the variance in the real concept method’s responses. High fidelity means the audit conclusions are about the real method, not about a bad approximation.

• Concept Faithfulness: This is the “does it actually matter?” test. If you perturb the region where a concept is supposedly detected, does the concept score actually change? If the model claims “vessel tortuosity drove my decision” but masking the vessel region doesn’t change the tortuosity score, the explanation is unfaithful. It’s the difference between someone saying “I stopped because of the red light” versus actually having stopped at the red light. Faithfulness is the causal link between the stated reason and the actual behavior.

Framework Shift

Before (mainstream):                  After (this paper):
                                      +--------------------------+
                                      | Independent Audit Layer  |
                                      +--------------------------+
                                           |           |
XAI Method --> Concept Explanation -->     v           v
              (trusted at face value)  Perturbation  Reliability
                                      Measurement    Scoring
                                           |           |
                                           v           v
                                      Surrogate   Faithfulness
                                      Fitting     Stability
                                                  Consistency

Approach: explanation --> accept       Approach: explanation --> test

From accepting concept explanations as ground truth to stress-testing them with perturbation-based auditing, the core shift is treating concept-based XAI outputs as claims to be verified rather than facts to be trusted.

Expert Assessment

Problem choice: This is a real and underexplored gap. The XAI community has been so focused on generating better explanations that the verification problem has lagged behind. As concept-based methods (CBMs, TCAV, prototype networks) get deployed in safety-critical domains like medical imaging, the question “is this explanation actually reliable?” becomes urgent. The timing is right — we’ve spent years building concept XAI methods and now need the quality assurance layer.

Method maturity: The approach is clever in its modularity — extending SMILE’s perturbation logic to concepts is a natural but non-trivial extension. However, the reliance on XGBoost as the surrogate raises questions: why XGBoost specifically? Would a simple linear model or a small neural network yield different fidelity scores? The locality weighting scheme is sensible but the paper could be more rigorous about sensitivity analysis on the weighting parameters. This is more “well-motivated engineering” than “surprising theoretical insight.”

Experimental integrity: The evaluation on retinal fundus images is appropriate and the comparison between MedSAM (spatial) and VLM (semantic) concept pathways is genuinely informative — showing that different concept types have different reliability profiles is a useful finding. The R-squared numbers (0.85 for MedSAM) are solid. One concern: the evaluation uses a single medical imaging domain. How sensitive are the reliability metrics to domain shift? The paper would be stronger with results on at least one non-medical domain to establish generalizability.

Writing quality: The paper reads clearly in its method description but the related work section is thin — it jumps from SMILE to the application without adequately positioning against other XAI auditing approaches (e.g., sanity checks for saliency maps, faithfulness metrics from the gradient-based literature). Section 4 (Results) would benefit from more qualitative examples showing *why certain concepts scored poorly on faithfulness — right now it’s mostly aggregate numbers without the storytelling that makes the paper memorable.

Verdict: weak accept — The gap is real, the method is reasonable, and the finding that concept reliability varies by pathway is genuinely useful. But the evaluation is narrow (one domain, one comparison) and the method’s novelty is incremental (extending SMILE to concepts rather than a fundamentally new framework).

Takeaways

Three things you can steal from this paper:

1. The auditing mindset for XAI: The most transferable idea is treating concept-based explanations as *claims to be verified, not outputs to be accepted. If you’re building any concept-based system (not just medical imaging), you can adopt this stress-test-then-trust workflow. Perturb your inputs, measure concept sensitivity, check for consistency.

2. The five-metric reliability scorecard: Attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency form a useful vocabulary for evaluating *any explanation method. You don’t need to use ConceptSMILE’s specific implementation — you can adopt the metric framework and apply it with simpler perturbation experiments in your own domain.

3. XGBoost as an explanation auditor: Using a simple surrogate model to approximate complex concept extraction behavior, then auditing the surrogate rather than the complex model directly, is a practical engineering pattern. It’s much easier to reason about an XGBoost’s behavior than a large segmentation model. This idea transfers to any situation where you need to verify a complex system’s explanations.