ClinSeekAgent: From Passive Evidence Consumption to Active Clinical Evidence Hunting

Paper: 2605.20176 Authors: Juncheng Wu, Letian Zhang, Yuhan Wang, Haoqin Tu, Hardy Chen, Zijun Wang, Cihang Xie, Yuyin Zhou Categories: cs.CL

The Gap

Existing clinical AI systems assume someone has already done the hard work: curating relevant evidence, extracting key facts from EHRs, selecting the right imaging studies, and packaging everything into a neat prompt. This works in controlled benchmarks but breaks in real clinical workflows where evidence is scattered across knowledge bases, raw EHR tables, imaging archives, and unstructured notes. Prior work like MedAgents and AgentClinic focuses on reasoning over pre-selected evidence, not on the evidence-seeking process itself. The gap: no one has built an agent that can start from a clinical query and raw data sources, then autonomously navigate, query, and synthesize multimodal evidence before reasoning.

Problem: Clinical decisions need evidence from scattered sources
    |
    v
Prior assumption: Evidence is pre-curated and handed to model
    |
    v
Reality gap: Real workflows require active evidence seeking
    |
    v
This paper's method: Agentic framework for dynamic evidence acquisition
    |
    v
Evidence: +3.2 F1 on EHR tasks, +15.1 on multimodal tasks (Claude Opus 4.6)
    |
    v
Conclusion: Active seeking > passive consumption for clinical reasoning

The Increment

One sentence: Before this paper, clinical AI consumed pre-selected evidence; after, it hunts for evidence across raw data sources like a clinician gathering information.

Core Mechanism

ClinSeekAgent operates in three phases. First, it decomposes a clinical query into sub-questions and decides which data sources to query (knowledge bases, EHR tables, imaging tools). Second, it executes those queries using specialized tools: a medical knowledge retriever for guidelines, a SQL-like EHR navigator for patient records, and vision models for chest X-rays. Third, it synthesizes the gathered evidence into a clinical decision, iteratively refining hypotheses as new information emerges.

The framework works both at inference time (augmenting frontier LLMs like Claude Opus) and training time (distilling agent trajectories into smaller models). At inference, the agent maintains a working memory of gathered evidence and dynamically plans next steps. At training, it generates high-quality trajectories that teach compact models to mimic the evidence-seeking process.

Clinical Query
    |
    v
[Planning Module] --> Decompose into sub-questions
    |                 Identify needed data sources
    v
[Tool Execution Layer]
    |
    +---> Medical Knowledge Base (guidelines, literature)
    |
    +---> EHR Navigator (SQL-like queries on raw tables)
    |
    +---> Imaging Tools (CXR analysis via vision models)
    |
    v
[Evidence Synthesis]
    |
    +---> Integrate findings
    |
    +---> Refine hypotheses
    |
    v
Clinical Decision

Think of ClinSeekAgent as a research assistant preparing a case presentation. You give them a patient question (“Does this patient have heart failure?”). They don’t wait for you to hand them a folder of relevant papers and test results. Instead, they go to the medical library (knowledge base), pull the patient’s chart from records (EHR navigator), walk down to radiology to review the chest X-ray (imaging tools), then return with a synthesis. The planning module is their mental checklist of what to investigate. The tool execution layer is their ability to navigate different hospital systems. The evidence synthesis is them sitting down to write the case summary, updating their thinking as each piece of evidence comes in.

Key Concepts

• Evidence-Seeking vs Evidence-Consumption: Traditional clinical AI is like a student taking an open-book exam where all relevant pages are bookmarked. Evidence-seeking AI is like a student who must first find the right textbooks, locate the relevant chapters, and extract key facts before answering. The former tests reasoning given information; the latter tests information retrieval, synthesis, and reasoning together. ClinSeekAgent shifts the paradigm from “here’s the evidence, now reason” to “find the evidence, then reason.”

• Agentic Trajectory Distillation: Large frontier models can perform complex multi-step reasoning but are expensive and slow. Smaller open-source models are fast but struggle with planning and tool use. Trajectory distillation bridges this gap: run the large model to generate high-quality evidence-seeking trajectories (sequences of tool calls, intermediate reasoning, evidence integration), then train the small model to imitate those trajectories. It’s like having a master chef record every decision while cooking a dish, then using that recording to train apprentices. The apprentice learns not just the final recipe but the adaptive process of tasting, adjusting, and improvising.

• Multimodal Evidence Integration: Clinical reasoning rarely relies on a single data type. A heart failure diagnosis might need EHR vitals (structured data), radiology reports (text), and chest X-rays (images). Multimodal integration means the agent must know when to invoke each modality, how to interpret outputs from different tools (SQL results vs vision model predictions), and how to weigh conflicting signals. ClinSeekAgent handles this by treating each modality as a tool with specific input/output contracts, then using the LLM’s reasoning to reconcile findings across modalities.

Framework Shift

Before (mainstream approach):        After (this paper):

[Pre-curated Evidence Package]       [Raw Data Sources]
         |                                  |
         v                                  v
    +--------+                         +---------+
    |  LLM   |                         | Agent   |
    |Reasoner|                         | Planner |
    +--------+                         +---------+
         |                                  |
         v                            +-----+-----+
    [Decision]                        |     |     |
                                      v     v     v
                                    [KB] [EHR] [IMG]
                                      |     |     |
                                      +-----+-----+
                                            |
                                            v
                                      [Synthesizer]
                                            |
                                            v
                                       [Decision]

Passive consumption                  Active acquisition
Single-step reasoning                Multi-step seeking
Fixed evidence set                   Dynamic evidence gathering

From curated-input reasoning to autonomous evidence-seeking, the core shift is delegating information retrieval and synthesis to the agent rather than the human curator.

Expert Assessment

Problem choice: This is a real gap. Clinical benchmarks like MIMIC-III and CheXpert provide clean, pre-processed data, but real EHRs are messy SQL tables and unstructured notes. The paper correctly identifies that evidence curation is a bottleneck in deploying clinical AI. However, the problem is somewhat narrow—evidence-seeking matters most in exploratory clinical scenarios, less so in routine workflows where evidence patterns are well-established.

Method maturity: The approach is pragmatic rather than novel. It combines existing techniques (tool-augmented LLMs, trajectory distillation, retrieval-augmented generation) and applies them to clinical data. The insight is in the application domain, not the method itself. The three-phase pipeline (plan, execute, synthesize) is standard agentic architecture. The contribution is showing this architecture works for clinical evidence-seeking, not inventing a new reasoning paradigm.

Experimental integrity: The baselines are fair. The paper compares against the same models with pre-curated evidence (Curated Input setting) and shows consistent gains from evidence-seeking. The ClinSeek-Bench construction is reasonable, pairing curated and raw-data versions of the same tasks. However, the evaluation is limited to retrospective tasks (predicting outcomes from historical data). Real clinical utility would require prospective validation where the agent’s evidence-seeking affects actual decisions. The distillation results (ClinSeek-35B-A3B) are impressive but only evaluated on AgentEHR-Bench, not the paper’s own ClinSeek-Bench.

Writing quality: The paper is well-structured but dense. The method section conflates inference-time and training-time usage, making it hard to track which components apply where. The results section would benefit from error analysis—when does evidence-seeking fail? The related work section is thorough but could better position this work relative to clinical decision support systems outside the LLM literature.

Verdict: Weak accept — solid execution on a real problem, but the method is incremental and evaluation stops short of clinical validation.

Takeaways

Practitioners can steal the three-phase agentic pattern (plan → execute → synthesize) for any domain where evidence is scattered across heterogeneous sources. The key insight: don’t wait for humans to curate data; teach the agent to navigate raw sources. The trajectory distillation pipeline is reusable—generate high-quality agent traces with a frontier model, then train a compact model to imitate them. This works beyond clinical AI: legal research (case law + statutes + filings), financial analysis (reports + market data + news), or scientific literature review (papers + datasets + code). The multimodal integration strategy (treat each data source as a tool with input/output contracts) generalizes to any multi-source reasoning task.