Paper: 2602.15019 Authors: Alisa Vinogradova, Vlad Vinogradov, Luba Greenwood, Ilya Yasny, Dmitry Kobyzev, Shoman Kasbekar, Kong Nguyen, Dmitrii Radkevich, Roman Doronin, Andrey Doronichev Categories: cs.AI, cs.IR
Abstract
The pharmaceutical innovation landscape has fundamentally shifted, with over 85% of patent filings now originating outside the United States and China accounting for nearly half of global filings. This paper addresses a critical gap in AI-powered drug asset discovery: existing deep research agents struggle to achieve high-recall, hallucination-free discovery across heterogeneous multilingual sources. The authors introduce Bioptic Agent, a tree-based self-learning system designed for complete drug asset scouting, alongside a rigorous benchmarking methodology. Testing against leading AI systems including Claude Opus 4.6, GPT-5.2 Pro, Perplexity Deep Research, Gemini 3 Pro, and Exa Websets, Bioptic Agent achieves 79.7% F1 score—substantially outperforming the next-best system at 56.2%. The work demonstrates that additional compute resources correlate with improved performance, validating compute-scaling approaches for complex discovery tasks.
Key Contributions
- Novel Benchmarking Framework: A multilingual multi-agent pipeline for constructing challenging completeness benchmarks using real queries from expert investors, BD professionals, and VCs as priors for conditional query generation
- Bioptic Agent Architecture: A tuned, tree-based self-learning agent specifically optimized for non-hallucinated, complete drug asset scouting across heterogeneous sources
- Comprehensive Evaluation: LLM-as-judge methodology calibrated to expert opinions for grading discovery completeness and accuracy
- Significant Performance Gains: 79.7% F1 score representing a 42% relative improvement over the strongest baseline (Claude Opus 4.6 at 56.2%)
- Compute-Performance Relationship: Empirical evidence that performance scales steeply with additional computational resources
The Global Drug Discovery Challenge
The pharmaceutical industry faces a critical information asymmetry problem. With China representing approximately 30% of global drug development activity spanning over 1,200 novel candidates, and the majority of patent filings occurring outside traditional U.S.-centric channels, investors and business development teams risk missing multi-billion-dollar opportunities. The challenge is compounded by:
Geographic Distribution: Over 85% of patent filings originate outside the U.S., with China alone accounting for nearly 50% of global patent activity. This represents a fundamental shift from historical patterns where U.S. sources dominated pharmaceutical innovation disclosure.
Linguistic Barriers: Many promising drug assets are disclosed primarily through regional, non-English channels including Chinese patent databases, regional regulatory filings, and local scientific publications. Traditional search methodologies optimized for English-language sources systematically miss these assets.
Information Heterogeneity: Drug asset information is scattered across diverse source types—patent databases, clinical trial registries, scientific literature, regulatory filings, company announcements, and conference proceedings—each with different structures, access methods, and disclosure patterns.
Speed Requirements: In competitive deal environments, being first to identify promising assets creates significant value. The time lag between asset disclosure and discovery by Western investors can represent the difference between securing favorable deal terms and missing opportunities entirely.
Current deep research AI agents, while powerful for many tasks, exhibit systematic weaknesses in this domain: incomplete coverage of non-English sources, hallucination of non-existent assets or incorrect details, and inability to achieve the high recall rates necessary for comprehensive scouting.
Methodology and Architecture
The authors develop a two-pronged approach: a rigorous benchmarking methodology to measure performance, and the Bioptic Agent system designed to excel on this benchmark.
Benchmark Construction
The benchmark addresses a fundamental challenge in evaluating drug asset discovery: creating realistic queries paired with verifiable ground-truth assets that reflect actual deal complexity. The methodology involves:
-
Expert Query Collection: Gathering real screening queries from professional investors, business development teams, and venture capital professionals actively engaged in pharmaceutical asset evaluation
-
Conditional Query Generation: Using expert queries as priors to generate benchmark queries that maintain realistic complexity and specificity while enabling systematic evaluation
-
Multilingual Ground Truth: Constructing ground-truth asset sets that are predominantly outside U.S.-centric radar, ensuring the benchmark tests ability to discover “under-the-radar” opportunities
-
Multi-Agent Pipeline: Employing a multilingual multi-agent system to validate ground truth and ensure completeness
Evaluation Framework
The grading system uses LLM-as-judge evaluation calibrated against expert human opinions. This approach balances scalability with accuracy, enabling systematic comparison across multiple systems while maintaining alignment with how domain experts would assess discovery quality. The evaluation focuses on:
- Recall: Proportion of ground-truth assets successfully discovered
- Precision: Accuracy of discovered assets (absence of hallucinations)
- F1 Score: Harmonic mean balancing recall and precision
Bioptic Agent Design
While the paper doesn’t exhaustively detail the architecture, key design principles include:
- Tree-Based Search: Structured exploration strategy that systematically covers the search space rather than relying on single-path retrieval
- Self-Learning: Adaptive mechanisms that improve performance through experience with the domain
- Multilingual Capability: Native support for processing and reasoning across multiple languages, particularly English and Chinese
- Hallucination Prevention: Explicit mechanisms to ground discoveries in verifiable sources and avoid generating false positives
The tree-based approach likely enables the agent to explore multiple hypotheses in parallel, backtrack when paths prove unproductive, and synthesize information across diverse sources—capabilities essential for comprehensive asset discovery.
Results and Performance Analysis
The experimental results reveal substantial performance gaps between systems:
| System | F1 Score | Performance Gap |
|---|---|---|
| Bioptic Agent | 79.7% | Baseline |
| Claude Opus 4.6 | 56.2% | -29.5% |
| Gemini 3 Pro + Deep Research | 50.6% | -36.5% |
| GPT-5.2 Pro | 46.6% | -41.5% |
| Perplexity Deep Research | 44.2% | -44.5% |
| Exa Websets | 26.9% | -66.2% |
Key Observations
The 79.7% F1 score achieved by Bioptic Agent represents a 42% relative improvement over the strongest baseline. This gap is particularly significant given that the baseline systems represent state-of-the-art deep research capabilities from leading AI labs.
The performance distribution suggests that general-purpose deep research agents, while capable, lack domain-specific optimizations necessary for pharmaceutical asset discovery. The gap between Claude Opus 4.6 (56.2%) and Gemini 3 Pro (50.6%) is relatively modest compared to the gap between these systems and Bioptic Agent, suggesting that domain specialization matters more than general model capabilities.
Compute Scaling
A critical finding is that Bioptic Agent’s performance improves steeply with additional computational resources. This validates the compute-scaling hypothesis: for complex discovery tasks, allocating more compute to thorough search and verification yields better results. This has important implications:
- Economic Trade-offs: Organizations can tune compute allocation based on the value of comprehensive discovery versus cost constraints
- Performance Ceiling: The steep scaling curve suggests current performance may not represent a fundamental ceiling, with room for improvement through additional compute
- Architectural Validation: The tree-based search architecture effectively utilizes additional compute, unlike approaches that might plateau
Error Analysis
While specific error patterns aren’t detailed in the abstract, the F1 scores suggest different failure modes across systems. The relatively low performance of Exa Websets (26.9%) suggests that pure retrieval-based approaches struggle with the complexity of pharmaceutical asset discovery. The moderate performance of general-purpose deep research agents (44-56%) indicates they achieve reasonable recall or precision individually but struggle to optimize both simultaneously.
Implications for Pharmaceutical Investment and Business Development
The research has immediate practical implications for how pharmaceutical investment and business development teams approach asset discovery:
Competitive Advantage: Organizations deploying specialized AI agents for asset scouting can achieve systematic advantages in deal flow. The 42% performance improvement translates directly to discovering more opportunities earlier in their lifecycle.
Risk Mitigation: Comprehensive scouting reduces the multi-billion-dollar risk of missing transformative assets. In an industry where single assets can represent $1B+ in value, even modest improvements in discovery completeness justify significant investment.
Resource Allocation: The compute-scaling relationship provides a framework for optimizing resource allocation. Teams can make informed decisions about when to invest additional compute for critical searches versus accepting faster but less complete results for routine monitoring.
Geographic Expansion: The demonstrated capability to effectively search non-U.S., multilingual sources enables Western investors and companies to systematically access the growing Chinese and Asian pharmaceutical innovation ecosystems.
Process Transformation: Moving from human-expert-driven scouting to AI-augmented discovery allows experts to focus on evaluation and deal execution rather than exhaustive search, potentially accelerating deal cycles.
Technical Insights and Future Directions
Several technical insights emerge from this work:
Domain Specialization Matters: The performance gap between Bioptic Agent and general-purpose systems demonstrates that domain-specific architecture and training provide substantial benefits. This suggests a broader principle: for high-stakes, specialized tasks, purpose-built agents outperform general-purpose systems even when the latter have larger scale or more sophisticated base models.
Tree-Based Search for Discovery: The tree-based approach appears particularly well-suited for discovery tasks requiring comprehensive coverage. Unlike linear search or single-path retrieval, tree-based methods can systematically explore the search space while managing computational costs.
Hallucination Prevention: Achieving high precision alongside high recall in a domain with verifiable ground truth requires explicit hallucination prevention mechanisms. This remains a critical challenge for AI systems in high-stakes applications.
Multilingual Reasoning: Effective pharmaceutical asset discovery requires genuine multilingual capability, not just translation. The system must understand domain concepts across languages and synthesize information from sources in different languages.
Future Research Directions
- Broader Domain Coverage: Extending beyond pharmaceuticals to other industries with similar geographic and linguistic discovery challenges
- Real-Time Monitoring: Adapting the approach for continuous monitoring rather than one-time searches
- Explainability: Developing methods to explain why specific assets were surfaced and how confidence levels are determined
- Human-AI Collaboration: Optimizing the interface between AI discovery and human expert evaluation
- Compute Efficiency: Investigating whether architectural improvements can achieve similar performance with reduced computational requirements
Takeaways
-
Geographic Shift in Innovation: Over 85% of pharmaceutical patent filings now originate outside the U.S., with China accounting for nearly half, creating systematic blind spots for traditional U.S.-centric discovery approaches.
-
Specialized Agents Outperform General Systems: Bioptic Agent’s 79.7% F1 score represents a 42% relative improvement over the best general-purpose deep research AI (Claude Opus 4.6 at 56.2%), demonstrating the value of domain-specific optimization.
-
Compute Scaling Validates Investment: Performance improves steeply with additional computational resources, providing a clear framework for organizations to optimize the trade-off between discovery completeness and cost.
-
Multilingual Capability is Essential: Effective drug asset discovery requires native multilingual processing, not just translation, to access the 30% of global drug development activity occurring in China and other non-English-speaking regions.
-
Benchmarking Methodology Advances the Field: The paper’s approach to constructing realistic benchmarks using expert queries as priors and LLM-as-judge evaluation calibrated to expert opinions provides a template for evaluating AI systems in other specialized domains.
-
High-Stakes Applications Demand Hallucination Prevention: Achieving 79.7% F1 requires balancing high recall with precision, demonstrating that preventing false positives is as critical as comprehensive discovery in pharmaceutical investment contexts.
-
Tree-Based Search Enables Comprehensive Coverage: The architectural choice of tree-based exploration appears particularly effective for discovery tasks requiring systematic coverage of heterogeneous, multilingual sources.
论文: 2602.15019 作者: Alisa Vinogradova, Vlad Vinogradov, Luba Greenwood, Ilya Yasny, Dmitry Kobyzev, Shoman Kasbekar, Kong Nguyen, Dmitrii Radkevich, Roman Doronin, Andrey Doronichev 分类: cs.AI, cs.IR
摘要
制药创新格局已发生根本性转变,目前超过85%的专利申请来自美国以外地区,中国占全球申请量的近一半。本文针对AI驱动的药物资产发现中的关键缺口:现有深度研究智能体难以在异构多语言来源中实现高召回率、无幻觉的发现。作者引入了Bioptic Agent,这是一个基于树的自学习系统,专为完整的药物资产侦察而设计,同时提出了严格的基准测试方法。在与包括Claude Opus 4.6、GPT-5.2 Pro、Perplexity Deep Research、Gemini 3 Pro和Exa Websets在内的领先AI系统的对比测试中,Bioptic Agent实现了79.7%的F1分数——大幅超越次优系统的56.2%。该研究证明额外的计算资源与性能提升相关,验证了复杂发现任务的计算扩展方法。
主要贡献
- 新颖的基准测试框架: 使用多语言多智能体管道构建具有挑战性的完整性基准,以来自专业投资者、商业拓展人员和风险投资家的真实查询作为先验进行条件查询生成
- Bioptic Agent架构: 专门优化的基于树的自学习智能体,用于跨异构来源的无幻觉、完整药物资产侦察
- 全面评估体系: 采用校准至专家意见的LLM评判方法,对发现完整性和准确性进行评分
- 显著的性能提升: 79.7%的F1分数相比最强基线(Claude Opus 4.6的56.2%)实现42%的相对改进
- 计算-性能关系: 实证证据表明性能随额外计算资源急剧提升
全球药物发现挑战
制药行业面临严重的信息不对称问题。中国约占全球药物开发活动的30%,涵盖超过1200个新型候选药物,而大多数专利申请发生在传统的以美国为中心的渠道之外,投资者和商业拓展团队面临错失数十亿美元机会的风险。这一挑战因以下因素而加剧:
地理分布: 超过85%的专利申请来自美国以外,仅中国就占全球专利活动的近50%。这代表了从历史上美国来源主导制药创新披露的根本性转变。
语言障碍: 许多有前景的药物资产主要通过区域性、非英语渠道披露,包括中国专利数据库、区域监管申报和本地科学出版物。针对英语来源优化的传统搜索方法系统性地遗漏了这些资产。
信息异构性: 药物资产信息分散在不同的来源类型中——专利数据库、临床试验注册、科学文献、监管申报、公司公告和会议论文集——每种都有不同的结构、访问方法和披露模式。
速度要求: 在竞争性交易环境中,率先识别有前景的资产创造显著价值。资产披露与西方投资者发现之间的时间差可能代表着获得有利交易条款与错失机会之间的区别。
当前的深度研究AI智能体虽然在许多任务上表现强大,但在该领域表现出系统性弱点:非英语来源覆盖不完整、幻觉出不存在的资产或错误细节,以及无法实现全面侦察所需的高召回率。
方法论与架构
作者开发了双管齐下的方法:严格的基准测试方法来衡量性能,以及旨在在该基准上表现出色的Bioptic Agent系统。
基准构建
该基准解决了评估药物资产发现的根本挑战:创建与可验证的真实资产配对的现实查询,反映实际交易复杂性。方法包括:
-
专家查询收集: 从积极从事制药资产评估的专业投资者、商业拓展团队和风险投资专业人士那里收集真实筛选查询
-
条件查询生成: 使用专家查询作为先验生成基准查询,保持现实的复杂性和特异性,同时实现系统评估
-
多语言真实数据: 构建主要在以美国为中心的雷达之外的真实资产集,确保基准测试发现”雷达下”机会的能力
-
多智能体管道: 采用多语言多智能体系统验证真实数据并确保完整性
评估框架
评分系统使用校准至专家人类意见的LLM评判评估。这种方法平衡了可扩展性与准确性,实现跨多个系统的系统比较,同时保持与领域专家评估发现质量方式的一致性。评估重点包括:
- 召回率: 成功发现的真实资产比例
- 精确率: 发现资产的准确性(无幻觉)
- F1分数: 平衡召回率和精确率的调和平均值
Bioptic Agent设计
虽然论文没有详尽描述架构,但关键设计原则包括:
- 基于树的搜索: 系统性覆盖搜索空间的结构化探索策略,而非依赖单路径检索
- 自学习: 通过领域经验改进性能的自适应机制
- 多语言能力: 原生支持跨多种语言(特别是英语和中文)的处理和推理
- 幻觉预防: 明确的机制将发现建立在可验证来源上,避免产生假阳性
基于树的方法可能使智能体能够并行探索多个假设,在路径证明无效时回溯,并跨不同来源综合信息——这些能力对全面资产发现至关重要。
结果与性能分析
实验结果揭示了系统之间的显著性能差距:
| 系统 | F1分数 | 性能差距 |
|---|---|---|
| Bioptic Agent | 79.7% | 基线 |
| Claude Opus 4.6 | 56.2% | -29.5% |
| Gemini 3 Pro + Deep Research | 50.6% | -36.5% |
| GPT-5.2 Pro | 46.6% | -41.5% |
| Perplexity Deep Research | 44.2% | -44.5% |
| Exa Websets | 26.9% | -66.2% |
关键观察
Bioptic Agent实现的79.7% F1分数相比最强基线代表42%的相对改进。考虑到基线系统代表了领先AI实验室的最先进深度研究能力,这一差距尤为显著。
性能分布表明,通用深度研究智能体虽然有能力,但缺乏制药资产发现所需的领域特定优化。Claude Opus 4.6(56.2%)和Gemini 3 Pro(50.6%)之间的差距相对较小,相比这些系统与Bioptic Agent之间的差距,表明领域专业化比通用模型能力更重要。
计算扩展
一个关键发现是Bioptic Agent的性能随额外计算资源急剧提升。这验证了计算扩展假设:对于复杂的发现任务,将更多计算分配给彻底搜索和验证会产生更好的结果。这具有重要意义:
- 经济权衡: 组织可以根据全面发现的价值与成本约束来调整计算分配
- 性能上限: 陡峭的扩展曲线表明当前性能可能不代表根本上限,通过额外计算仍有改进空间
- 架构验证: 基于树的搜索架构有效利用额外计算,不像某些可能达到平台期的方法
错误分析
虽然摘要中没有详细说明具体的错误模式,但F1分数表明不同系统存在不同的失败模式。Exa Websets相对较低的性能(26.9%)表明纯检索方法难以应对制药资产发现的复杂性。通用深度研究智能体的中等性能(44-56%)表明它们单独实现合理的召回率或精确率,但难以同时优化两者。
对制药投资和商业拓展的影响
该研究对制药投资和商业拓展团队如何进行资产发现具有直接的实践意义:
竞争优势: 部署专业AI智能体进行资产侦察的组织可以在交易流中获得系统性优势。42%的性能改进直接转化为在生命周期早期发现更多机会。
风险缓解: 全面侦察降低了错失变革性资产的数十亿美元风险。在单个资产可能代表10亿美元以上价值的行业中,即使发现完整性的适度改进也证明了重大投资的合理性。
资源分配: 计算扩展关系为优化资源分配提供了框架。团队可以就何时为关键搜索投资额外计算与接受更快但不太完整的常规监控结果做出明智决策。
地理扩张: 展示的有效搜索非美国、多语言来源的能力使西方投资者和公司能够系统性地访问不断增长的中国和亚洲制药创新生态系统。
流程转型: 从人类专家驱动的侦察转向AI增强的发现,使专家能够专注于评估和交易执行而非详尽搜索,可能加速交易周期。
技术洞察与未来方向
从这项工作中浮现出几个技术洞察:
领域专业化很重要: Bioptic Agent与通用系统之间的性能差距表明,领域特定的架构和训练提供了实质性好处。这暗示了一个更广泛的原则:对于高风险、专业化任务,专用智能体优于通用系统,即使后者具有更大规模或更复杂的基础模型。
基于树的搜索用于发现: 基于树的方法似乎特别适合需要全面覆盖的发现任务。与线性搜索或单路径检索不同,基于树的方法可以系统性地探索搜索空间,同时管理计算成本。
幻觉预防: 在具有可验证真实数据的领域中实现高精确率和高召回率需要明确的幻觉预防机制。这仍然是高风险应用中AI系统的关键挑战。
多语言推理: 有效的制药资产发现需要真正的多语言能力,而不仅仅是翻译。系统必须理解跨语言的领域概念,并综合来自不同语言来源的信息。
未来研究方向
- 更广泛的领域覆盖: 扩展到具有类似地理和语言发现挑战的其他行业
- 实时监控: 调整方法用于持续监控而非一次性搜索
- 可解释性: 开发方法解释为什么浮现特定资产以及如何确定置信度水平
- 人机协作: 优化AI发现与人类专家评估之间的接口
- 计算效率: 研究架构改进是否能以减少的计算需求实现类似性能
要点总结
-
创新的地理转移: 超过85%的制药专利申请现在来自美国以外,中国占近一半,为传统的以美国为中心的发现方法创造了系统性盲点。
-
专业智能体优于通用系统: Bioptic Agent的79.7% F1分数相比最佳通用深度研究AI(Claude Opus 4.6的56.2%)实现42%的相对改进,展示了领域特定优化的价值。
-
计算扩展验证投资: 性能随额外计算资源急剧提升,为组织优化发现完整性与成本之间的权衡提供了清晰框架。
-
多语言能力至关重要: 有效的药物资产发现需要原生多语言处理,而非仅仅翻译,以访问发生在中国和其他非英语地区的30%全球药物开发活动。
-
基准测试方法推进领域发展: 论文使用专家查询作为先验构建现实基准以及校准至专家意见的LLM评判评估的方法,为评估其他专业领域的AI系统提供了模板。
-
高风险应用需要幻觉预防: 实现79.7% F1需要平衡高召回率与精确率,表明在制药投资背景下,防止假阳性与全面发现同样关键。
-
基于树的搜索实现全面覆盖: 基于树的探索的架构选择对于需要系统性覆盖异构、多语言来源的发现任务似乎特别有效。