Can LLM Agents Autonomously Hack IoT Devices? VEXAIoT Says Yes

Paper: 2607.09653 Authors: Katherine Swinea, Kshitiz Aryal, Lopamudra Praharaj, Maanak Gupta Categories: cs.CR, cs.AI

The Gap

Existing research has explored LLM agents for general penetration testing and Capture-the-Flag (CTF) challenges — think PentestGPT, AutoPentest, and similar frameworks that treat hacking as a generic reasoning task. But these systems assume standard enterprise IT environments with known protocols, plentiful documentation, and relatively clean attack surfaces. IoT is a different beast entirely: constrained devices running stripped-down firmware, obscure network services, non-standard configurations, and hardware-specific quirks that make textbook exploit chains useless.

Nobody had systematically asked: can LLM agents handle the unique attack surface of IoT — where a Raspberry Pi running a vulnerable MQTT broker behaves nothing like a web server with a known CVE? The gap is specific: the intersection of LLM-driven autonomous exploitation and IoT-specific vulnerability classes (per OWASP IoT Top 10) was uncharted territory.

[Problem]                    [Assumption]                 [Method]
IoT devices are        -->   LLM agents can          -->   VEXAIoT: two-agent
inherently vulnerable        reason about IoT-             framework with
(unique attack surface,      specific attack               detection agent
constrained hardware,        surfaces given                 + execution agent
obscure protocols)           tool access

                                                        |
                                                        v

[Conclusion]                 [Evidence]
LLM-driven IoT           <-- 95% success across 10    <--
exploitation is              scenarios in IoTGoat
feasible in controlled       and Metasploitable2
settings, but real-world     (260 total executions,
generalization remains       avg <2min per attack)
an open question

The Increment

One sentence: Before this paper, nobody had built and evaluated an LLM-agent system specifically designed to autonomously discover and exploit IoT vulnerabilities; after it, we have a working prototype showing 95% success on simulated IoT attack surfaces with a clean two-agent architecture.

Core Mechanism

VEXAIoT splits the attack workflow into two specialized LLM agents that communicate sequentially. The Vulnerability Detection Agent handles reconnaissance: it scans the target environment, identifies running services, queries for known vulnerability patterns, and produces a structured report of potential attack vectors. Think of it as the intelligence analyst who maps the terrain before anyone moves in.

The Attack Execution Agent takes that intelligence report and plans concrete exploit sequences. It has access to offensive security tools — Metasploit modules, network scanners, exploit scripts — and orchestrates them step-by-step. After each step, it evaluates whether the attack succeeded, failed, or produced partial results that require adaptation. This agent is the field operative who reads the brief and executes.

Data flows linearly: target environment → detection agent (scanning + analysis) → vulnerability report → execution agent (planning + tool orchestration) → exploit results → feedback loop for retry or escalation. Both agents use LLM reasoning to handle ambiguity — when a scan returns unexpected results or an exploit partially fails, the LLM doesn’t just retry; it reasons about what went wrong and adjusts.

+-----------------+     scan results     +---------------------+
|  Target IoT Env | --------------------> | Detection Agent     |
| (IoTGoat,       |                      | - Service discovery |
|  Metasploitable)|                      | - Vuln fingerprint  |
+-----------------+                      | - OWASP mapping     |
                                         +----------+----------+
                                                    |
                                            vuln report
                                                    |
                                                    v
                                         +---------------------+
                                         | Execution Agent     |
                                         | - Attack planning   |
                                         | - Tool orchestration|
                                         | - Result evaluation |
                                         +----------+----------+
                                                    |
                                             exploit results
                                                    |
                                                    v
                                            success / failure
                                            feedback to retry

Here’s the metaphor that makes this click: imagine a two-person safecracking team. The first person is the lock analyst — she walks around the vault, examines the lock mechanism, checks for known vulnerabilities in the model, and writes a technical brief: “This is a Sargent & Greenleaf 6730, known to have a manipulation vulnerability in the fence mechanism at the 3-4-5 gate pattern.” The second person is the operative — he reads the brief, selects his tools (stethoscope, tension wrench, pick set), and executes the specific sequence. If the first attempt doesn’t work, he doesn’t randomly try things; he reports back (“the contact points shifted at L45, suggesting the wheel pack is different”) and the analyst refines the diagnosis.

VEXAIoT works the same way. The detection agent is the lock analyst — identifying what’s running on the target, what known vulnerabilities apply, and producing a structured attack brief. The execution agent is the operative — taking that brief, selecting Metasploit modules or custom scripts, executing step-by-step, and feeding results back. The LLM’s role is the “reasoning” that both team members share: the ability to interpret ambiguous signals and adapt, rather than just following a fixed playbook.

Key Concepts

• Multi-agent decomposition: Instead of asking one giant LLM to “hack this IoT device,” you split the job into two agents with different expertise. This is like how a hospital separates diagnosis (radiologist reads the scan) from treatment (surgeon performs the operation). Each agent gets a focused prompt, accesses only its relevant tools, and produces structured output that the other agent can consume. The benefit: each agent’s context window isn’t polluted with irrelevant information, and you can swap out either agent independently.

• OWASP IoT Top 10 mapping: The Open Web Application Security Project maintains a list of the ten most critical IoT vulnerability categories — things like insecure default passwords, lack of firmware update mechanisms, insecure network services. VEXAIoT explicitly maps each attack scenario to one of these categories. This isn’t just organization for organization’s sake; it means the detection agent’s “mental model” of what to look for is structured around a known taxonomy rather than free-form scanning. It’s the difference between a doctor who checks symptoms systematically vs. one who just guesses.

• Controlled environment evaluation: Both IoTGoat and Metasploitable2 are intentionally vulnerable systems designed for security training. They’re the sparring dummies of cybersecurity. This is important context: the 95% success rate means “LLM agents can exploit vulnerabilities we specifically designed to be exploitable,” which is a different claim than “LLM agents can exploit real-world IoT devices.” The paper is transparent about this, but readers should calibrate expectations accordingly.

Framework Shift

Before (mainstream approach):          After (this paper):
+----------------+                    +------------------+
| Human          |                    | Detection Agent  |
| PenTester      |                    | (LLM + scanning) |
| - manual recon |                    |    |             |
| - tool scripting|                   |    | vuln report  |
| - exploit      |                    |    v             |
|   debugging    |                    | Execution Agent  |
| - reporting    |                    | (LLM + tools)    |
+----------------+                    | - auto-planning  |
        |                             | - auto-execution |
        v                             +------------------+
  Slow, expensive,                         |
  depends on skill                         v
                                   Fast (avg <2min),
                                   consistent,
                                   but simulated only

From manual, skill-dependent penetration testing to autonomous agent-driven exploitation, the core shift is treating IoT hacking as a reasoning and orchestration task that LLMs can decompose and execute — not as an art requiring years of human expertise.

Expert Assessment

Problem choice: Real gap, well-motivated. IoT security testing is genuinely under-automated, and the intersection with LLM agents is a natural frontier. That said, the paper’s positioning as “unexplored” is slightly overstated — there’s prior work on automated exploit generation and IoT fuzzing that doesn’t use LLMs but addresses similar goals. The novelty is in the LLM-agent framing, not in the concept of automated IoT exploitation per se.

Method maturity: This is more “clever integration” than “fundamental insight.” The two-agent architecture is straightforward and well-structured, but there’s no novel algorithmic contribution — it’s prompt engineering plus tool orchestration. A simpler approach (single agent with a well-designed tool-use prompt) might achieve similar results, and the paper doesn’t test that ablation. The agent architecture feels like the minimum viable design rather than a deeply optimized one.

Experimental integrity: This is where I have concerns. Both test environments (IoTGoat, Metasploitable2) are *designed to be vulnerable — they’re training platforms. The 95% success rate is impressive in context, but the ceiling is high by construction. There’s no comparison against: (1) existing automated tools like Nmap + Metasploit scripting, (2) a single-agent baseline, (3) a non-LLM rule-based system. Without these baselines, we can’t tell how much the LLM reasoning actually contributes vs. just having a well-configured attack toolkit. The 10 attack scenarios is also a small number. Token usage and latency metrics are welcome additions, though.

Writing quality: Solid structure, clear motivation, good experimental detail. The related work section is thin — it could much better position against the non-LLM automated exploitation literature (e.g., cyber reasoning systems from DARPA CGC). The threat model is never explicitly stated: what’s the attacker’s capability level, and what defenses exist? A brief section on limitations (beyond the conclusion paragraph) would strengthen credibility.

Verdict: weak accept — solid proof-of-concept with a clean design, but the evaluation lacks baselines and real-world grounding that would make the 95% success rate meaningful beyond training environments.

Takeaways

Three things worth stealing:

1. The two-agent decomposition pattern (detect → execute) is immediately transferable to any domain where you can separate “intelligence gathering” from “action execution.” Think automated code review (one agent identifies vulnerabilities, another generates patches) or automated trading (one agent scans signals, another executes trades).

2. Mapping to a structured vulnerability taxonomy (OWASP Top 10) is a practical way to constrain an LLM agent’s search space. If you’re building agents for any domain with a known classification system, explicitly injecting that taxonomy into the detection agent’s prompt produces more organized, auditable output.

3. The evaluation methodology — even though the environments are simulated, the paper shows how to structure multi-scenario IoT testing with clear success/failure criteria. If you’re benchmarking any automated security tool, this is a reasonable template to adapt.