FinHarness: Inline Safety Monitoring for Finance LLM Agents

Paper: 2605.27333 Authors: Haoxuan Jia, Yang Liu, Bin Chong, Yingguang Yang, Yancheng Chen, Jiayu Liang, Qian Li, Hanning Lu, Kefu Xu, Hao Zheng Categories: cs.CL

The Gap

Finance LLM agents execute multi-step workflows involving irreversible operations like fund transfers. Existing safety approaches fall into two camps: boundary filters that check only the initial prompt (missing malicious drift mid-trajectory), and post-hoc auditing that reviews the entire trace after execution completes (too late to prevent damage, and computationally expensive for long traces). Neither approach can catch a prompt injection that starts benign but pivots to unauthorized actions halfway through a 10-step workflow.

Problem: Malicious drift in multi-step agent execution
   |
   v
Assumption: Safety decisions need per-step context + cumulative risk
   |
   v
Method: Inline monitors (Query + Tool) + Adaptive routing (Cascade)
   |
   v
Evidence: ASR 38.3% -> 15.0%, benign approval 41.1% -> 39.3%
   |
   v
Conclusion: Real-time intervention beats boundary/post-hoc filtering

The Increment

One sentence: Before this paper, you either checked safety once at the boundary or audited everything after the fact; now you can monitor each step inline and route expensive verification only when risk accumulates.

Core Mechanism

FinHarness wraps the agent with three components. The Query Monitor tracks intent drift across conversation turns, comparing the current query against the original request to detect semantic pivots. The Tool Monitor evaluates each prospective tool call before execution, checking whether the action aligns with stated intent and business rules. The Cascade module maintains a running risk score: when accumulated risk crosses a threshold, it routes the decision to an advanced LLM judge; otherwise, a lightweight judge suffices.

When a risk factor fires (intent drift detected, suspicious tool call, high cumulative risk), FinHarness doesn’t just block—it re-injects the risk evidence back into the agent’s input as ex-ante context. The agent sees “Warning: detected intent drift from ‘check balance’ to ‘transfer funds’” and can refuse, re-plan, or justify why the action is legitimate. This turns the safety system from a gatekeeper into a feedback loop.

User Query --> [Query Monitor] --> Risk Score
                      |                 |
                      v                 v
                Agent Reasoning --> [Tool Monitor] --> Risk Score
                      |                 |                 |
                      v                 v                 v
                Tool Call --> [Cascade Router] --> Lightweight Judge
                                      |                 |
                                      +---> Advanced Judge (if risk > threshold)
                                              |
                                              v
                                        Risk Evidence --> Re-inject to Agent

Think of FinHarness as a co-pilot with a hand on the emergency brake. The Query Monitor is the navigator checking if you’re still heading to the original destination. The Tool Monitor is the safety officer reviewing each maneuver before you execute it. The Cascade is the decision protocol: for routine checks, the safety officer handles it; for high-stakes calls, you escalate to the chief compliance officer. When something looks wrong, instead of just slamming the brakes, the co-pilot taps your shoulder and says “Hey, you said we’re going to the bank, but you just turned toward the casino—want to explain?” You (the agent) can then course-correct or justify the detour.

Key Concepts

• Intent Drift: Imagine you ask your assistant to “check my account balance.” Midway through, a prompt injection in a retrieved document tricks the assistant into thinking you now want to “transfer $10,000 to account XYZ.” The original intent (read-only query) has drifted to a different intent (write operation). Intent drift detection compares the semantic meaning of the current action against the initial request. If the distance exceeds a threshold, it flags potential hijacking. Concrete example: initial query embedding is close to “information retrieval,” current tool call embedding is close to “financial transaction”—cosine distance jumps from 0.1 to 0.7, triggering a warning.

• Inline Monitoring: Traditional safety checks happen at two points: before the agent starts (input filter) or after it finishes (output audit). Inline monitoring inserts checkpoints *during execution, evaluating each intermediate step. It’s the difference between checking a student’s homework only at submission versus walking around the classroom and catching mistakes as they happen. For agents, this means intercepting each tool call, running a safety check, and deciding whether to allow, block, or escalate before the tool executes.

• Adaptive Routing (Cascade): Not all decisions need the same level of scrutiny. Checking account balance is low-risk; transferring funds is high-risk. A cascade uses a lightweight judge (fast, cheap) for most cases and escalates to an advanced judge (slow, expensive) only when risk signals accumulate. Think of airport security: most passengers go through the quick metal detector; only flagged passengers get the full pat-down. FinHarness maintains a running risk score across the trajectory—if it stays low, the lightweight judge handles all checks; if it spikes, the advanced judge takes over.

Framework Shift

Before (boundary + post-hoc):        After (inline + adaptive):

  Input Filter                         Query Monitor (per turn)
       |                                      |
       v                                      v
  [Agent runs freely]                   Agent Step 1
       |                                      |
       v                                      v
  [Agent runs freely]                   Tool Monitor --> Cascade
       |                                      |              |
       v                                      v              v
  [Agent runs freely]                   Agent Step 2    Lightweight
       |                                      |          or Advanced
       v                                      v              |
  Output Audit                          Tool Monitor <-------+
  (too late)                                  |
                                              v
                                        Risk Re-injection

From “check once at the gate, audit the corpse” to “monitor every step, escalate only when needed,” the core shift is real-time intervention with cost-aware verification.

Expert Assessment

Problem choice: Real gap. Finance agents are moving from demos to production, and the “filter input, pray, audit output” paradigm doesn’t cut it for irreversible operations. The paper targets a genuine pain point where existing safety research (mostly focused on single-turn refusal) hasn’t caught up to multi-step agentic workflows.

Method maturity: Solid engineering, not a breakthrough. The components (semantic drift detection, per-step monitoring, tiered verification) are individually known techniques. The contribution is the integration and the adaptive routing strategy. The risk re-injection idea is clever—turning the safety system into a feedback loop rather than a hard gate. No simpler approach is being overlooked; this is the right level of complexity for the problem.

Experimental integrity: Baselines are fair (boundary filter, post-hoc audit, always-advanced judge). The FinVault benchmark is purpose-built for this paper, which raises reproducibility questions—would be stronger with evaluation on an external dataset. The 4.7× cost reduction claim is credible given the cascade design, but the paper doesn’t report latency, which matters for production deployment. The benign approval drop (41.1% → 39.3%) is small but non-zero; the paper doesn’t deeply investigate false positives.

Writing quality: The abstract and intro are crisp. The method section is dense—Figure 2 (system architecture) does heavy lifting, but the text doesn’t walk through a concrete example end-to-end until the appendix. Rewriting Section 3.2 (Cascade module) with a step-by-step trace would make the paper much more accessible. The related work section is thorough but reads like a literature dump; tighter framing around “why existing approaches fail for multi-step agents” would sharpen the narrative.

Verdict: Weak accept — Addresses a real problem with a well-engineered solution and solid empirical results, but the contribution is incremental (integration of known techniques) and the evaluation is limited to a single, paper-specific benchmark.

Takeaways

Adaptive verification routing: The cascade pattern (lightweight judge by default, escalate on accumulated risk) is directly transferable to any domain where you need to balance safety and cost. Healthcare agents, legal document processing, code generation—anywhere you have a spectrum of risk and can’t afford to run expensive checks on every step.

Risk re-injection as feedback: Instead of blocking silently or logging for post-hoc review, inject detected risks back into the agent’s context. This turns safety from a binary gate into a reasoning aid. The agent can self-correct, explain, or escalate. Applicable beyond LLM agents—think of it as “show your work” for automated systems.

Per-step monitoring beats boundary filtering: For multi-step workflows, checking only the initial input is insufficient. This is obvious in hindsight but underappreciated in practice. If you’re building agents that execute sequences of actions, instrument each action with a safety check. The cost is worth it for irreversible operations.