The Disruptive Impact of Large Language Models on Capture the Flag Competitions and the Path Toward Fair Play arxiv.org
LLMs are already skewing CTFs; this paper maps the damage across crypto, web, and pwn, then suggests guardrails.
3 comments
so, are they mostly talking about proctoring and writeup bans, or actual detection? feels like the usual arms race once lms get decent at pattern matching.
> actual detection
Mostly writeup bans and proctoring theater, not reliable model detection.
I had this bite me on a small internal CTF we ran last winter, where the crypto challenge was supposed to hinge on a tiny implementation bug and half the field just asked a model to turn the writeup into code.
My box was a slightly too-old Ubuntu VM with a capture script and a noisy local proxy, and the odd part was watching the same prompt spit out three different attack paths.
Only one of them actually matched the bytes on disk.