A controlled study of how far a security-tool catalog carries an LLM agent, where that capability stops, and what targeted engineering recovers. 774 trials on picoCTF, three clients, before and after a round of fixes.
This repository is a fork of HexStrike-AI, used as the subject of a research project. We measured how well the orchestrator carries an LLM through security challenges, fixed the weak points the measurement exposed, and measured the effect of those fixes. The fork keeps the upstream architecture and tool set; everything in this README is our addition on top of it.
| Testbed | 86 picoCTF challenges (27 Easy, 31 Medium, 28 Hard) across 7 categories |
| Design | 3 tool-access regimes x 3 model/client setups = 774 trials |
| Models | Claude (Sonnet 4.6) via Claude Code; DeepSeek (deepseek-chat) via RooCode and 5ire |
| Headline | Overall solve rate 55.4% to 72.0%, every configuration improved |
| Significance | Non-overlapping 95% Wilson intervals; paired McNemar p < 0.001 |
| Biggest lever | Prompting and re-pointing existing tools, not new tools (see attribution below) |
- The fixes worked, and the gain is statistically clean. Overall solve rate rose from 55.4% to 72.0% across 774 trials. Every configuration improved, with baseline and post-fix 95% Wilson confidence intervals that do not overlap and a paired McNemar test at p < 0.001.
- The client mattered as much as the difficulty. On the same DeepSeek model, RooCode solved 59.7% and 5ire solved 27.9%, a 2.1x gap with nothing changed but the client that drove the model.
- Good prompting and re-pointing existing tools beat building new ones. Of 128 recovered trials, 79 came from general behavioral and default fixes and 25 from re-pointing the existing pwntools tool, against 24 from every newly built tool combined. The single largest lever was one tool-description rewrite plus a one-line prompt nudge.
- "HexStrike-only" was a soft constraint, not a clean partition. Under the two regimes that forbid native tools, agents still made 2,908 native
execute_commandcalls and 717Bashcalls, heaviest on the two weaker clients. - The server shipped defects a test harness would have caught. Two advertised tools did not exist (so every call failed), one tool was dead because a second registration shadowed it, and the server bound every network interface while ignoring its own configured loopback default.
- The Original System
- The Team and Why We Did This
- What the Baseline Showed
- What We Changed
- Results
- Repository Structure
- Running and Reproducing
- Documentation
- Citation
- License and Attribution
HexStrike-AI is an open-source orchestrator that exposes a large set of security tools to an LLM agent over the Model Context Protocol (MCP). A Flask server wraps each tool as an HTTP route, and a FastMCP layer presents those routes to the model as callable tools. The agent plans, calls a tool, reads its output, and chains tools toward a goal such as solving a CTF challenge or assessing a target. The version we forked exposes 169 MCP tools backed by 176 server routes over 150+ external Kali utilities.
This fork keeps the upstream architecture and tool set. Installation steps and the full tool catalog are documented in the upstream repository, and this fork runs the same way. Everything described below is our addition.
We are master's students at the Suzhou Institute for Advanced Research, University of Science and Technology of China. This work is a research project on LLM-driven security tooling, carried out by Romain Gerard and Assmaa Zeghaider under the supervision of Yan Guo. We treated HexStrike-AI as something to study and improve rather than a product to ship. The question was practical: how far does a tool catalog carry an LLM agent, where does that capability stop, and can targeted engineering move the line.
We ran 86 picoCTF challenges (27 Easy, 31 Medium, 28 Hard) across seven categories, under three tool-access regimes and three model/client setups, for 774 trials in total. The regimes were free solve (any tool, native or HexStrike), HexStrike-ranked (HexStrike tools with a preference ordering), and HexStrike-strict (HexStrike tools only). The setups were Claude (Sonnet 4.6) through Claude Code, and DeepSeek (deepseek-chat) through two clients, RooCode and 5ire. Holding the model fixed across the two DeepSeek clients let us separate the client's effect from the model's.
The baseline run surfaced the following:
- Capability dropped steeply with difficulty, from 75.7% on Easy to 30.6% on Hard, and the drop held inside almost every category. The hardest cells were General/Hard and Web/Hard.
- The client driving the model mattered as much as the difficulty did. On the same DeepSeek model, RooCode solved 59.7% and 5ire solved 27.9%, a 2.1x gap with nothing changed but the client.
- The two regimes that forbid native tools were not honored. Across them the agents still made 2,908 native
execute_commandcalls and 717Bashcalls, heaviest on the two weaker clients. A "HexStrike-only" number was therefore a soft constraint, not a clean partition. - Nine challenges failed in every configuration and every regime. These marked capabilities the baseline tooling did not cover: multi-step web chains, disk forensics, and harder cryptography.
- Two tools the server advertised,
web_requestandsource_code_read, did not exist, so every call to them failed. One other tool was dead because a second registration shadowed it, and the server bound every network interface while ignoring its own configured loopback default. - Success detection was hardcoded to the picoCTF flag format in many places, which limited how much of the behavior would carry to real targets.
The work had three parts: fixes to existing tools and agent behavior, eleven new capability tools, and a pass that decoupled success detection from CTF conventions.
- Input validation and automatic dependency installation for the Python execution tool.
- Session, redirect, and JSON handling for the HTTP testing tool.
- Defaults for port scanning suited to the challenge setting.
- A description rewrite that points binary work at pwntools. This single change recovered more challenges than any other fix.
- A boolean-blind SQL injection extractor.
- Post-processing for oversized forensics output.
- Removal of the two advertised tools that did not exist.
- A bind fix so the server uses its configured loopback default instead of opening every interface.
On top of these, four modifiers apply to every run: a hypothesis-first strategy preamble, a phased decomposition for Hard challenges, a per-call confidence and next-tool signal, and a HexStrike-only constraint injected through the 5ire proxy.
Each tool wraps an external capability aimed at a failure we saw in the baseline.
| Tool | What it does | Target challenge(s) |
|---|---|---|
rsa_factor |
Recovers keys from weak or smooth RSA moduli | Crypto: Very Smooth |
compression_oracle |
Builds a CRIME/BREACH byte-by-byte recovery harness | Crypto: Compress and Attack |
timing_oracle |
Recovers a secret character by character from response-time or instruction-count differences | Forensics: SideChannel |
sqli_order_oracle |
Extracts data via ORDER BY / CASE WHEN boolean-blind injection | Web: ORDER ORDER |
evtx_parser |
Parses Windows .evtx logs and surfaces notable entries | Forensics: Event-Viewing |
smb_ipp_exploit |
Enumerates SMB shares and IPP/CUPS printers and reads files for secrets | General: Printer Shares 2 and 3 |
blockchain_exploit |
Drives the Foundry cast CLI (call/send/storage) for access-control, overflow, and reentrancy challenges |
Blockchain: Access Control, Smart Overflow, Reentrance |
| ROP-chain builder | Generates a ROP chain (ROPgadget) and a pwntools exploit template | Binary / Hard |
| Disk-image mount | Parses a disk image with The Sleuth Kit, listing allocated and deleted files | Forensics: DISKO 3, UnforgottenBits |
| Encrypted-PCAP decryptor | Decrypts a captured session with tshark using a supplied key (TLS/RSA/WPA) | Forensics: WebNet0, WebNet1 |
| Headless XSS/CSRF chainer | Injects an XSS/CSRF payload and drives a headless browser, capturing DOM, cookies, and alerts | Web: noted, secure-email-service |
We replaced the hardcoded picoCTF flag format with one configurable detection layer. It keeps the picoCTF prefix by default, so the running evaluation was not affected, and it also recognizes real-world indicators: cloud and API keys, GitHub and Slack tokens, JSON Web Tokens, private-key headers, and credential assignments. The new tools and the fixes were checked against real, non-CTF targets to confirm they run there, though we did not measure solve rate outside picoCTF.
After applying the changes we re-ran the trials that had not succeeded at baseline, 289 of them, and set aside 56 as out of reach (challenges that would need a substantial new tool, external infrastructure, or infeasible effort to move). Re-runs used the same models, clients, and prompts as the baseline. Overall solve rate rose from 55.4% to 72.0%. Every configuration improved, with baseline and post-fix 95% Wilson confidence intervals that do not overlap and a paired McNemar test at p < 0.001.
Baseline outcomes by configuration:
| Configuration | Solved | Failed | Partial | Rate |
|---|---|---|---|---|
| Claude / Sonnet 4.6 | 203 | 39 | 16 | 78.7% |
| DeepSeek / RooCode | 154 | 95 | 9 | 59.7% |
| DeepSeek / 5ire | 72 | 172 | 14 | 27.9% |
| Overall | 429 | 306 | 39 | 55.4% |
| Configuration | Baseline | Post-fix | Change |
|---|---|---|---|
| Claude / Sonnet 4.6 | 78.7% | 89.9% | +11.2 pp |
| DeepSeek / RooCode | 59.7% | 76.4% | +16.7 pp |
| DeepSeek / 5ire | 27.9% | 49.6% | +21.7 pp |
| Overall | 55.4% | 72.0% | +16.6 pp |
The regime breakdown shows the client effect from another angle. Claude sits near its ceiling under all three regimes and RooCode lifts roughly evenly, while 5ire shows a strong regime effect, weakest when tool preference is only ranked and strongest when native tools are removed entirely.
The gradient stayed monotonic after the fixes, with the largest lift in the middle tier and the smallest at Hard.
| Difficulty | Before | After | Change |
|---|---|---|---|
| Easy | 75.7% | 94.2% | +18.5 pp |
| Medium | 60.2% | 81.7% | +21.5 pp |
| Hard | 30.6% | 39.7% | +9.1 pp |
128 previously-failed trials now pass. Of these, 79 came from the general behavioral and default fixes, 25 from re-pointing the existing pwntools tool, and 24 from the tools we built (22 from the eleven new tools, 2 from the added SQL injection extractor). The one description-level pwntools change accounts for more recoveries than all the tools we built combined.
For seven of the eleven new tools, recorded calls and recoveries break down as below; the other four supplied the remaining two recoveries. The zero for timing_oracle is a signal-bound challenge, not a tool that failed to run.
| Tool | Calls | Recoveries |
|---|---|---|
rsa_factor |
2 | 2 |
compression_oracle |
10 | 4 |
timing_oracle |
14 | 0 |
smb_ipp_exploit |
30 | 7 |
sqli_order_oracle |
10 | 2 |
evtx_parser |
17 | 3 |
blockchain_exploit |
10 | 2 |
| Total | 93 | 20 |
The two heatmaps share a scale, so the post-fix panel reads as the same landscape shifted upward. The lift is broad rather than concentrated in one category.
To check that single-run verdicts are stable, we re-ran ten challenges three times on each DeepSeek client under the free-solve regime, 60 runs in total. RooCode agreed with itself on all ten. 5ire agreed on seven of ten, with the three that varied sitting on borderline Medium challenges and splitting two-to-one with a clear majority. Across both clients, 17 of 20 were unanimous.
| Configuration | Unanimous (of 10) | Agreement |
|---|---|---|
| DeepSeek / RooCode | 10 | 100% |
| DeepSeek / 5ire | 7 | 70% |
| Overall | 17/20 | 85% |
hexstrike-eval/
├── hexstrike_server.py Flask tool server (our fixes + the 11 new capability tools live here)
├── hexstrike_mcp.py FastMCP layer that exposes the routes to the agent
├── prompt_generator.py Builds the per-challenge prompts (baseline task + intervention scaffolding)
├── hexstrike-ai-mcp.json MCP client configuration
├── requirements.txt Python dependencies
├── check_deps.sh Checks for the external Kali tools the server shells out to
├── make_figures.py Regenerates every figure from the recorded numbers
├── results/ Baseline per-trial logs (LLM / Client / Category / Difficulty / Challenge)
├── results_with_fixes/ Post-fix re-run logs
├── variance_results/ Run-to-run reliability sub-study (60 runs)
├── figures/ Generated charts (PNG + PDF + EPS) and the architecture diagram
├── hooks/ Tool-call logging and the 5ire LLM proxy used to inject the strict constraint
├── docs/ Analysis, paper source, and engineering audits (see below)
└── papers/ Reading notes for the related-work survey
Each leaf directory under results/ and results_with_fixes/ holds the prompt sent and the transcript recorded for one challenge, following LLM/Client/Category/Difficulty/Challenge/. For example, results/Claude/Sonnet4.6/Web/Easy/Cookies/.
Full installation, including the external Kali utilities the tools shell out to, is documented in the upstream repository. This fork runs the same way:
git clone https://github.com/0x4m4/hexstrike-ai # or this fork once public
python3 -m venv hexstrike_env
source hexstrike_env/bin/activate
python3 -m pip install -r requirements.txt
python3 hexstrike_server.py # starts the Flask tool server on :8888Then point an MCP client at the server by editing the path and address in hexstrike-ai-mcp.json. Run ./check_deps.sh to see which external tools are present on the host.
The per-trial logs are already committed, so the results can be inspected without re-running anything. To regenerate the figures from the recorded numbers:
python3 -m pip install matplotlib numpy
python3 make_figures.py # writes PNG + PDF + EPS into figures/make_figures.py does not recompute anything from the logs; every value is taken from docs/results_analysis.md, which traces each number back to the run that produced it.
The docs/ directory carries the full write-up and the engineering trail behind it:
- docs/results_analysis.md — every headline number with the run it came from (baseline, post-fix, inferential statistics, and the variance sub-study).
- docs/architecture.md — how the server, the MCP layer, and the agent loop fit together.
- docs/features.md — the tool catalog and what each capability covers.
- docs/code_health_audit.md — the defects the baseline exposed, including the dead and non-existent tools.
- docs/improvement_plan.md — the planned changes and the reasoning behind each.
A paper describing this study is under review. The full citation will be added here on publication. In the meantime, please cite:
@misc{hexstrike-eval,
title = {Determinants and Limits of LLM Security-Tool Orchestration: A Study with HexStrike-AI},
author = {Gerard, Romain and Zeghaider, Assmaa and Guo, Yan},
year = {2026},
note = {Manuscript under review}
}This fork inherits the upstream MIT license. Upstream HexStrike-AI is by Muhammad Osama (0x4m4); the base orchestration, the wrapped tools, and the MCP architecture are theirs. The evaluation design, the fixes and capability tools, and the logging and measurement layer are ours.