PCAS: Policy Compiler for Secure Agentic Systems

Paper: 2602.16708 Authors: Nils Palumbo, Sarthak Choudhary, Jihye Choi, Prasad Chalasani, Mihai Christodorescu, Somesh Jha Categories: cs.CR, cs.AI, cs.MA

Abstract

As LLM-based agents are deployed in increasingly sensitive contexts—customer service, approval workflows, data access control, and regulatory compliance—the need for robust authorization mechanisms becomes critical. Traditional approaches that embed policies directly in prompts offer no enforcement guarantees, leaving systems vulnerable to policy violations through prompt manipulation or model reasoning failures. This paper introduces PCAS (Policy Compiler for Agentic Systems), a framework that provides deterministic policy enforcement for multi-agent systems through compile-time instrumentation and runtime monitoring.

PCAS addresses the fundamental challenge that linear message histories cannot capture complex information flows across agents. Instead, it models system state as a dependency graph that tracks causal relationships among events including tool calls, results, and inter-agent messages. Policies are expressed in a Datalog-derived declarative language that naturally handles transitive information flow and cross-agent provenance tracking. A reference monitor intercepts all actions before execution, blocking policy violations deterministically regardless of model behavior. Evaluated across three domains—prompt injection defense, pharmacovigilance workflows, and customer service—PCAS improves compliance from 48% to 93% with zero violations in instrumented runs.

Key Contributions

• A dependency graph model for tracking information flow in multi-agent systems that captures causal relationships beyond linear message histories
• A Datalog-derived policy language for expressing complex authorization rules including transitive information flow and cross-agent provenance
• A compile-time instrumentation approach that transforms existing agent implementations into policy-compliant systems without security-specific restructuring
• A reference monitor architecture that provides deterministic policy enforcement independent of LLM reasoning capabilities
• Empirical validation across three case studies demonstrating 93% policy compliance with zero violations in instrumented systems

Technical Architecture

PCAS operates through a three-stage pipeline: policy compilation, system instrumentation, and runtime enforcement.

The policy compilation stage translates high-level policy specifications into executable enforcement rules. Policies are written in a Datalog-derived language that extends traditional access control with information flow tracking. For example, a policy preventing customer data leakage might specify: forbid(send_email(Agent, Recipient, Content)) :- contains_pii(Content), not authorized_recipient(Recipient). The compiler analyzes these rules to identify enforcement points and generates monitoring code.

System instrumentation wraps existing agent implementations with tracking and enforcement logic. PCAS intercepts three critical event types: tool invocations, tool results, and inter-agent messages. Each event is recorded in the dependency graph with edges representing information flow. When Agent A calls a tool that returns data subsequently used by Agent B, the graph captures this transitive dependency. The instrumentation is transparent to the original agent code—no manual security refactoring is required.

Runtime enforcement uses a reference monitor that evaluates policy rules against the current dependency graph before allowing actions to execute. When an agent attempts an action, the monitor queries the graph to determine if the action would violate any policy. For information flow policies, this involves checking whether sensitive data would reach unauthorized destinations through transitive paths. The monitor blocks violations deterministically, providing security guarantees independent of model behavior.

Dependency Graph Model

The core innovation enabling PCAS is its dependency graph representation of system state. Traditional agent frameworks maintain linear conversation histories that lose critical information about causality and data provenance. PCAS constructs a directed acyclic graph where nodes represent events and edges represent information flow.

Three node types capture the essential events in agentic systems:

• Tool call nodes represent agent decisions to invoke external functions
• Tool result nodes capture the data returned by tool executions
• Message nodes represent communication between agents or with users

Edges in the graph encode how information flows between events. When an agent makes a tool call, an edge connects from the agent’s context to the call node. When the tool returns results, an edge connects to a result node. If those results inform a subsequent message or tool call, additional edges capture that dependency. This structure naturally represents transitive information flow: if data from source $S$ influences event $E_1$, which influences $E_2$, the graph contains a path $S \rightarrow E_1 \rightarrow E_2$.

The graph model enables sophisticated policy queries. To determine if an action would leak sensitive data, the monitor performs a reachability analysis: does a path exist from a sensitive data source to the proposed action? For approval workflows, the monitor checks whether required authorization events appear as ancestors of the proposed action in the graph. This graph-based reasoning handles complex scenarios that would be difficult or impossible to express with linear history inspection.

Evaluation Results

PCAS was evaluated across three case studies representing different policy enforcement challenges: information flow control for prompt injection defense, approval workflows in pharmacovigilance, and organizational policies for customer service.

In the prompt injection defense scenario, policies prevent agents from executing commands derived from untrusted user input. Without PCAS, frontier models (GPT-4, Claude) exhibited 48% policy compliance—they frequently executed injected commands despite prompt-based warnings. With PCAS instrumentation, compliance reached 93% with zero policy violations in the subset of runs where the agent attempted policy-violating actions. The remaining 7% represented cases where agents failed to complete tasks for reasons unrelated to policy enforcement.

The pharmacovigilance case study involved a multi-agent system processing adverse drug event reports. Policies required supervisor approval before submitting reports to regulatory authorities. Baseline agents bypassed approval workflows 35% of the time, either through reasoning failures or prompt manipulation. PCAS enforcement achieved 100% compliance—no reports were submitted without proper authorization chains visible in the dependency graph.

Customer service evaluation tested organizational policies like “don’t offer refunds exceeding $X without manager approval” and “don’t access customer data from other regions.” Across 200 test scenarios with varying complexity, uninstrumented agents violated policies in 52% of cases. PCAS-instrumented systems achieved 93% compliance with zero violations in runs where policy-relevant actions were attempted. The 7% gap represented task failures unrelated to security.

Implications and Future Directions

PCAS demonstrates that deterministic policy enforcement for agentic systems is achievable without sacrificing the flexibility that makes LLM-based agents valuable. The compile-time instrumentation approach means developers can build agents using familiar frameworks and patterns, then add security as a separate concern through policy specification.

The dependency graph model has implications beyond security. It provides a foundation for explainability—showing users why an action was blocked by tracing information flow through the graph. It enables auditing by maintaining a complete provenance record of how decisions were made. It could support debugging by helping developers understand unexpected agent behavior through causal analysis.

Several directions for future work emerge. The current policy language, while expressive, could be extended with temporal operators for policies involving time-based constraints. The system could be optimized for lower overhead through incremental graph updates rather than full recomputation. Integration with formal verification tools could provide static guarantees about policy properties before deployment.

The evaluation also reveals limitations. PCAS cannot prevent all security issues—it enforces specified policies but cannot detect unspecified vulnerabilities. Policy specification requires security expertise, though the declarative language is more accessible than imperative security code. The system adds runtime overhead, though measurements show this is acceptable for most applications (typically <100ms per action).

Takeaways

1. Embedding authorization policies in prompts provides no enforcement guarantees—LLMs can fail to follow instructions through reasoning errors or prompt manipulation
2. Dependency graphs capture information flow in multi-agent systems more accurately than linear message histories, enabling sophisticated policy queries about transitive data flow
3. Datalog-derived policy languages provide a natural fit for expressing authorization rules that account for provenance and information flow across agents
4. Compile-time instrumentation allows adding deterministic policy enforcement to existing agent implementations without security-specific restructuring
5. Reference monitor architectures can provide security guarantees independent of LLM behavior by intercepting and validating actions before execution
6. Empirical results show PCAS improves policy compliance from 48% to 93% across frontier models with zero violations in instrumented runs
7. The approach generalizes across diverse policy types including information flow control, approval workflows, and organizational access restrictions