LLM Novice Uplift on Dual-Use, In Silico Biology Tasks

Paper: 2602.23329 Authors: Chen Bo Calvin Zhang, Christina Q. Knight, Nicholas Kruus, Jason Hausenloy, Pedro Medeiros, Nathaniel Li, Aiden Kim, Yury Orlovskiy, Coleman Breen, Bryce Cai Categories: cs.AI, cs.CL, cs.CR, cs.CY, cs.HC

Abstract

This paper presents a rigorous empirical investigation into whether large language models genuinely uplift novice users on complex biological tasks with dual-use implications. While LLMs demonstrate impressive performance on biology benchmarks, the critical question of whether they enable non-experts to perform tasks previously requiring specialized training has remained largely unexplored. The researchers conducted a multi-model, multi-benchmark study comparing novices with LLM access against those with internet-only access across eight biosecurity-relevant task sets. The results are striking: LLM-assisted novices achieved 4.16 times higher accuracy than controls (95% CI [2.63, 6.87]) and outperformed internet-only experts on three out of four benchmarks. Notably, standalone LLMs often exceeded human-LLM collaborative performance, suggesting users struggle to fully leverage model capabilities. Despite existing safeguards, 89.6% of participants reported minimal difficulty accessing dual-use information, highlighting urgent biosecurity concerns.

Key Contributions

• First large-scale human uplift study measuring LLM impact on novice performance in biosecurity-relevant biological tasks
• Quantitative demonstration of substantial uplift: novices with LLMs achieved 4.16x accuracy improvement over internet-only controls
• Evidence that LLM-assisted novices can match or exceed expert-level performance on complex in silico biology tasks
• Discovery that standalone LLMs often outperform human-LLM collaboration, indicating suboptimal human prompting strategies
• Documentation that current safeguards provide minimal barriers to dual-use information access (89.6% success rate)
• Establishment of a methodological framework for interactive uplift evaluations beyond static benchmarks

Experimental Design and Methodology

The study employed a rigorous between-subjects design with participants randomly assigned to either LLM-access or internet-only control conditions. Eight distinct task sets were developed spanning biosecurity-relevant domains, including protein engineering, synthetic biology pathway design, and computational virology. Tasks were designed to be complex enough to require substantial expertise while remaining tractable for motivated novices given sufficient time and resources.

Participants were given generous time allocations—up to 13 hours for the most demanding tasks—to ensure that performance differences reflected genuine capability gaps rather than time pressure artifacts. The LLM-access group could use multiple state-of-the-art models, while controls had unrestricted internet access including scientific databases, literature, and online tools. This design choice was deliberate: the goal was to measure whether LLMs provide uplift beyond what comprehensive internet resources already offer.

The researchers implemented multiple quality control measures, including attention checks, expert validation of task difficulty, and post-hoc analysis of participant strategies. Baseline expert performance was established for four of the eight benchmarks using domain specialists with internet-only access, providing crucial reference points for interpreting novice performance.

Results and Performance Analysis

The quantitative results reveal substantial and statistically significant uplift. The primary finding—a 4.16x accuracy improvement (95% CI [2.63, 6.87])—represents a dramatic capability enhancement. This effect size is particularly noteworthy given that control participants had full internet access, suggesting LLMs provide capabilities beyond information retrieval and synthesis.

Perhaps most striking is the comparison with expert baselines. On three of four benchmarks where expert performance data was available, LLM-assisted novices matched or exceeded expert accuracy. This finding challenges conventional assumptions about the irreplaceability of domain expertise for complex technical tasks. The one benchmark where experts maintained superiority involved highly specialized knowledge of regulatory frameworks, suggesting that LLMs may be less effective for tasks requiring deep contextual understanding of institutional structures.

The performance gap between standalone LLMs and human-LLM collaboration presents an intriguing puzzle. In many cases, the models alone achieved higher accuracy than when assisting human users. This suggests that novices may lack the prompting sophistication, domain knowledge to validate outputs, or strategic thinking needed to fully leverage model capabilities. The implication is that current uplift measurements may actually underestimate the potential risk if users develop more effective interaction strategies.

Regarding safeguards, the results are concerning from a biosecurity perspective. Despite existing content filters and safety measures, 89.6% of participants reported encountering minimal friction when seeking dual-use information. This high success rate suggests that current mitigation strategies are insufficient for preventing misuse by motivated actors.

Implications for Biosecurity and AI Safety

The findings carry profound implications for biosecurity policy and AI governance. The demonstrated ability of LLMs to uplift novices to expert-level performance on sensitive biological tasks represents a qualitative shift in dual-use risk landscape. Previously, certain capabilities were effectively gatekept by the years of training required to develop relevant expertise. LLMs appear to substantially lower this barrier.

The research highlights a critical gap in current AI safety evaluations. Traditional benchmarks measure model capabilities in isolation but fail to capture the interactive, human-in-the-loop dynamics that characterize real-world use. An LLM might perform poorly on a static benchmark while still providing dangerous uplift to malicious actors through iterative refinement and strategic prompting.

The ineffectiveness of current safeguards demands urgent attention. Content filtering and refusal training appear insufficient when users can iteratively probe models, rephrase queries, or use indirect approaches. More sophisticated mitigation strategies may be needed, potentially including usage monitoring, capability throttling for sensitive domains, or mandatory authentication and audit trails.

The finding that standalone LLMs often outperform human-LLM collaboration suggests a complex risk trajectory. As users develop better prompting strategies and mental models of LLM capabilities, uplift effects may increase substantially. This implies that current measurements may represent a lower bound on potential dual-use risks.

Takeaways

1. LLMs provide dramatic uplift to novices on complex biological tasks, with a 4.16x accuracy improvement over internet-only access, fundamentally altering the accessibility landscape for specialized capabilities.

2. LLM-assisted novices can match or exceed expert performance on most biosecurity-relevant benchmarks, suggesting that years of specialized training may no longer serve as an effective barrier to certain dual-use capabilities.

3. Current AI safeguards are largely ineffective, with 89.6% of participants reporting minimal difficulty accessing dual-use information, highlighting urgent need for more robust mitigation strategies.

4. Standalone LLMs often outperform human-LLM collaboration, indicating that users are not yet fully leveraging model capabilities—a concerning finding given that improved prompting strategies could increase dual-use risks.

5. Traditional static benchmarks are insufficient for assessing dual-use risks; interactive uplift evaluations that measure human-AI collaborative performance are essential for comprehensive safety assessment.

6. The research establishes a methodological framework for rigorous uplift studies, emphasizing the importance of generous time allocations, expert baselines, and ecologically valid task design in evaluating real-world AI impacts.