DeepSight: An All-in-One LM Safety Toolkit

Paper: 2602.12092 Authors: Bo Zhang, Jiaxuan Guo, Lijun Li, Dongrui Liu, Sujin Chen, Guanxu Chen, Zhijie Zheng, Qihao Lin, Lewen Yan, Chen Qian Categories: cs.CL, cs.AI, cs.CR, cs.CV

Abstract

As large language models (LLMs) and multimodal large language models (MLLMs) become increasingly powerful, ensuring their safety has become a critical priority. Current safety workflows suffer from fragmentation—evaluation, diagnosis, and alignment are handled by separate tools that don’t communicate effectively. Safety evaluation typically identifies external behavioral risks without uncovering internal root causes, while safety diagnosis often remains disconnected from concrete risk scenarios. This paper introduces DeepSight, an open-source project that unifies safety evaluation and diagnosis into a cohesive paradigm. DeepSight consists of two core components: DeepSafe for evaluation and DeepScan for diagnosis. By establishing unified task and data protocols, DeepSight transforms safety assessment from opaque black-box testing into transparent white-box mechanistic analysis, making it the first open-source toolkit to support both frontier AI risk evaluation and integrated safety evaluation-diagnosis workflows.

Key Contributions

• Integrated Paradigm: Proposes a novel evaluation-diagnosis integrated approach that bridges the gap between external behavioral assessment and internal mechanistic understanding
• Dual Toolkit Architecture: Introduces DeepSafe for comprehensive safety evaluation and DeepScan for deep mechanistic diagnosis, with unified protocols enabling seamless integration
• White-Box Transformation: Converts traditional black-box safety evaluation into white-box insights by connecting evaluation results with internal model mechanisms
• Frontier AI Support: First open-source toolkit to support frontier AI risk evaluation, addressing emerging safety challenges in cutting-edge models
• Scalability and Efficiency: Designed as a low-cost, reproducible, and highly scalable solution for large-scale model safety assessment

The Fragmentation Problem in Current Safety Workflows

Traditional large model safety pipelines operate in silos. Safety evaluation tools can identify when a model produces harmful outputs—detecting toxicity, bias, or dangerous content—but they cannot explain why these failures occur at a mechanistic level. Conversely, interpretability and diagnosis tools may analyze model internals but often lack grounding in concrete safety scenarios and real-world risk contexts.

This fragmentation creates several critical issues. First, safety alignment efforts proceed without clear understanding of which internal mechanisms need modification, potentially leading to over-correction that degrades general capabilities. Second, evaluation results cannot inform targeted interventions because the causal chain from internal representations to external behaviors remains obscured. Third, the lack of standardized protocols between evaluation and diagnosis stages prevents systematic iteration and improvement.

DeepSight addresses these challenges by establishing a unified framework where evaluation and diagnosis share common task definitions, data formats, and analytical protocols. This integration enables practitioners to trace safety failures from observable behaviors back to specific model components and forward from mechanistic insights to predicted behavioral outcomes.

Architecture and Methodology

DeepSight’s architecture consists of two complementary toolkits working in concert:

DeepSafe (Evaluation Toolkit) provides comprehensive safety assessment across multiple dimensions. It implements standardized evaluation protocols for both LLMs and MLLMs, covering traditional safety concerns (toxicity, bias, harmful content generation) as well as frontier AI risks (deception, power-seeking, situational awareness). DeepSafe generates structured evaluation reports that not only flag safety violations but also categorize them according to risk type, severity, and contextual factors.

DeepScan (Diagnosis Toolkit) performs deep mechanistic analysis to uncover the internal causes of safety failures. Rather than treating models as black boxes, DeepScan employs interpretability techniques to examine attention patterns, activation spaces, and information flow pathways. It identifies which model components (specific layers, attention heads, or neuron clusters) are responsible for generating unsafe outputs in particular contexts.

The key innovation lies in the unified protocol layer connecting these toolkits. When DeepSafe identifies a safety violation, it generates a structured report with standardized metadata. DeepScan can then automatically load this report, retrieve the relevant model states and inputs, and perform targeted analysis on the specific failure mode. This bidirectional flow—from evaluation to diagnosis and back—enables iterative refinement where diagnostic insights inform more precise evaluation strategies.

Implications for AI Safety Research

DeepSight’s integrated approach has significant implications for the broader AI safety research community. By open-sourcing both evaluation and diagnosis capabilities with unified protocols, it establishes a foundation for reproducible safety research. Researchers can now systematically investigate how specific architectural choices, training procedures, or alignment techniques affect both external safety behaviors and internal model mechanisms.

The toolkit’s support for frontier AI risks is particularly timely. As models become more capable, new safety challenges emerge—models that can strategically deceive evaluators, exhibit goal-directed behavior beyond their training objectives, or demonstrate concerning levels of self-awareness. DeepSight provides the infrastructure to evaluate these advanced risks while simultaneously diagnosing their mechanistic origins.

Furthermore, the white-box transformation enabled by DeepSight’s integrated paradigm opens new avenues for targeted safety interventions. Rather than applying broad alignment techniques that may inadvertently harm model capabilities, practitioners can use diagnostic insights to design surgical interventions that address specific safety mechanisms while preserving general functionality.

Takeaways

1. DeepSight unifies safety evaluation and diagnosis into a single integrated framework, addressing the fragmentation problem in current large model safety workflows
2. The toolkit consists of DeepSafe for comprehensive evaluation and DeepScan for mechanistic diagnosis, connected through unified task and data protocols
3. By transforming black-box safety assessment into white-box mechanistic analysis, DeepSight enables more targeted and effective safety interventions
4. DeepSight is the first open-source toolkit to support both frontier AI risk evaluation and integrated evaluation-diagnosis workflows
5. The unified protocol architecture enables reproducible safety research and systematic iteration between behavioral assessment and mechanistic understanding