From 15% to 93%: AI Agents Are Getting Scary Good at Cybersecurity

In August 2024, the best AI models solved between 12.5% and 17.5% of tasks on Cybench, a benchmark of professional hacking challenges. By February 2026, frontier models reached 93% on the same benchmark, and AI systems were finding hundreds of previously unknown vulnerabilities in real software.
The Cybench Benchmark
- Creator: Stanford’s Center for Research on Foundation Models (CRFM), August 2024.
- Tasks: 40 professional capture-the-flag (CTF) challenges from HackTheBox, SekaiCTF, Glacier and HKCert.
- Difficulty: the first human teams needed between 2 minutes and nearly 25 hours to solve each task.
Success rates at launch:
- Claude 3.5 Sonnet: 17.5%;
- GPT-4o: 12.5%;
- Claude 3 Opus and OpenAI o1-preview: 10%;
- Gemini 1.5 Pro and Llama 3.1 405B: 7.5%.
At launch, models rarely solved tasks that took the first human team more than 11 minutes.
The Jump to 93%
- Claude Opus 4.6 averaged 93% on Cybench, according to its February 2026 system card.
- A March 2026 paper from the UK AI Security Institute (AISI) states that “frontier models now reach 93% on Cybench — up from 17.5% at launch.”
- Anthropic describes the benchmark as “saturated”, meaning it no longer separates the strongest models.
- Security firm Irregular recorded Cybench scores of about 10% in early 2024 and 82% in November 2025.
UK AI Security Institute Findings
AISI’s Frontier AI Trends Report, published December 18, 2025, covers more than 30 frontier systems tested since November 2023:
- Apprentice-level tasks: success rose from under 9% in late 2023 to about 50% in 2025.
- Expert-level tasks: the first model to complete any task requiring about 10 years of human experience appeared in 2025.
- Task length: the length of cyber tasks AI can complete autonomously grew from under 10 minutes in early 2023 to over an hour by mid-2025.
Later AISI evaluations in 2026:
- Claude Mythos Preview (April 2026): succeeded on 73% of expert-level CTF tasks.
- Simulated network attack: Mythos Preview completed a 32-step corporate network attack, estimated at 20 hours of expert work, in 3 of 10 attempts.
- GPT-5.5 (April 2026): average pass rate of 71.4% on expert-level tasks.
- Cost: GPT-5.5 solved a reverse-engineering task in 10 minutes 22 seconds for $1.73 in API costs; an expert tester needed about 12 hours.
Finding Real Vulnerabilities
- Google Big Sleep: built by Google DeepMind and Project Zero, it reported its first 20 vulnerabilities on August 4, 2025, including flaws in FFmpeg and ImageMagick.
- XBOW: on June 24, 2025, became the first AI system to reach No. 1 on HackerOne’s U.S. leaderboard, after submitting nearly 1,060 vulnerability reports.
- Claude Opus 4.6: found more than 500 previously unknown high-severity flaws in open-source software, including Ghostscript and OpenSC, as reported in February 2026.
- Google CodeMender: by October 2025, it had submitted 72 security fixes to open-source projects totalling more than 4.5 million lines of code.
DARPA’s AI Cyber Challenge
The final of the AI Cyber Challenge (AIxCC) took place at DEF CON on August 8, 2025:
- Team Atlanta won $4 million, Trail of Bits $3 million and Theori $1.5 million.
- Finalist systems found 54 of 63 planted vulnerabilities (86%) and patched 43 (68%).
- They also discovered 18 real, previously unknown vulnerabilities across more than 54 million lines of code.
- An average patch took about 45 minutes and cost about $152 per task.
AI vs Human Hackers
- In Palisade Research’s AI vs. Humans CTF in March 2025, 4 of 6 AI teams solved 19 of 20 challenges and placed in the top 5% of 152 teams.
- Similar head-to-head results are emerging in physical tasks, where a robot ping-pong paddle beats elite players.
Misuse and Limits
- In November 2025, Anthropic reported that GTG-1002, a group it assessed as Chinese state-sponsored, used Claude Code against about 30 targets, with AI performing 80–90% of the tactical work.
- The International AI Safety Report 2026 states that AI systems still “cannot reliably execute long, multi-stage attack sequences” without humans.
- AISI notes that its benchmarks do not predict performance against defended real-world systems.
Network Security Online
Security vendors, threat-intelligence feeds and network tooling projects frequently operate on the .net extension, which was introduced in 1985 for network-related organisations. Registering a dedicated .net domain is common for vulnerability disclosure pages and security advisories.
Key Numbers at a Glance
- 17.5%: best Cybench score at launch in August 2024.
- 93%: Cybench score of Claude Opus 4.6 in February 2026.
- 73%: expert-level CTF tasks solved by Claude Mythos Preview.
- 500+: high-severity flaws found by Claude Opus 4.6.
- $4 million: AIxCC first prize won by Team Atlanta.






