filtered-shellcode picoCTF 2021 Solution

Published: April 2, 2026

Description

A program that reads shellcode and runs it - but before executing, it rewrites your input by inserting NOP (0x90) bytes between your instruction bytes. In effect only some of your bytes survive intact, so you must lay out your shellcode in short instructions (with the inserted NOPs falling on instruction boundaries) so it still executes correctly and spawns a shell.

Download the binary and examine how it rewrites your input before running it.

bash
wget https://mercury.picoctf.net/static/.../fun
bash
chmod +x fun
bash
nc mercury.picoctf.net <PORT_FROM_INSTANCE>

Solution

Want to try it yourself first?

The guided walkthrough reveals hints one step at a time.

Walk me through it
This is a shellcoding challenge, not a buffer overflow: the program runs the bytes you provide directly, but it interleaves NOPs into them first. The Buffer Overflow Binary Exploitation guide covers the shellcode and execve("/bin/sh") fundamentals reused here.
  1. Step 1
    Reverse the loader to see how it mangles your shellcode
    Observation
    I noticed the binary is named 'fun' and the description says it rewrites input before executing it, which suggested the transform itself (not a blocklist) was the key obstacle and that I needed to reverse the loader in Ghidra to determine the exact NOP-insertion pattern before crafting any shellcode.
    Open the binary in Ghidra. main reads your shellcode and, if its length is odd, appends one NOP (0x90) to make it even. The execute function then copies your bytes into an executable buffer but inserts two NOP bytes after every two of your bytes. So in memory your shellcode becomes: [byte][byte][0x90][0x90][byte][byte][0x90][0x90]... Confirm this layout by single-stepping in GDB.
    bash
    # Ghidra: look at main (the odd-length NOP pad) and execute (the NOP interleave).
    bash
    gdb -q ./fun
    bash
    (gdb) break *<addr just before the call to execute>
    bash
    (gdb) run
    bash
    # feed an obvious marker like 'pqrst' (each a 1-byte instruction)
    bash
    (gdb) x/16bx $eax    # observe: pq 90 90 rs 90 90 t ...
    What didn't work first

    Tried: Assume the filter is a blocklist and try sending shellcode that avoids common bad bytes like 0x00 or 0x0a.

    The transform is not a blocklist at all - it is a byte insertion pass that splices two NOP bytes after every two of your bytes. Sending standard shellcode that merely avoids null bytes still results in opcodes being split mid-instruction. You need to reverse the loader logic in Ghidra first to understand the exact transform before crafting anything.

    Tried: Skip Ghidra and test in GDB by setting a breakpoint after main returns, then inspecting the buffer.

    main returns after calling execute, at which point the shellcode has already run (or crashed). You need a breakpoint just before the call to execute - not after main returns - so you can inspect the transformed bytes in memory before the CPU tries to execute them. The marker byte trick (sending printable ASCII letters as 1-byte instructions) shows the NOP layout at that early breakpoint.

    Learn more

    This is the whole gimmick: the loader does not reject bytes, it injects 0x90 (NOP) bytes between yours. After every two of your bytes, two NOPs are spliced in. A NOP does nothing and is one byte, so each pair of NOPs is just dead space the CPU slides through.

    The consequence: any instruction longer than two bytes gets torn apart, because NOPs land in the middle of its opcode and operands. Only instructions that fit in the two-byte windows survive intact. Your job is to write the whole payload out of short instructions that tolerate the inserted NOPs on their boundaries.

  2. Step 2
    Build the execve shellcode out of short instructions
    Observation
    I noticed from reversing the loader that two NOPs are inserted after every two of my bytes, which meant any instruction longer than two bytes would be split mid-opcode and suggested I needed to rewrite the standard execve('/bin/sh') payload using only 1- and 2-byte instructions to keep NOP insertions on clean boundaries.
    Rewrite a standard 32-bit execve('/bin/sh', 0, 0) so every instruction is short enough to survive. The problem instructions are the multi-byte pushes of the '/bin/sh' constant. Replace each 5-byte push of a 4-byte immediate with a sequence that builds the constant a byte at a time: zero a register, then repeatedly (mov a byte into the low 8 bits, add to the accumulator, shift left 8). Those mov/add/shl forms are each ~2 bytes, so the inserted NOPs fall harmlessly between them. Where a single push is unavoidably odd-aligned, pad with one NOP yourself so the splice lands on an instruction boundary.
    bash
    ; 32-bit execve('/bin/sh', 0, 0) rebuilt from short instructions (sketch).
    ; Goal: get '/bin/sh\x00' onto the stack and into ebx without any >2-byte push.
    xor eax, eax        ; 2 bytes
    xor ecx, ecx        ; 2 bytes
    push eax            ; null terminator (1 byte) + pad with a NOP if needed
    ; build "n/sh" and "//bi" in eax one byte at a time, push each:
    xor eax, eax
    mov cl, 0x68        ; 'h'  -> 2 bytes
    add eax, ecx        ; 2 bytes
    shl eax, 8          ; 2 bytes
    ; ...repeat mov cl,<char> / add / shl for each byte of the chunk...
    push eax
    ; (do the second 4-byte chunk the same way)
    mov ebx, esp        ; ebx -> "/bin/sh"
    xor ecx, ecx        ; argv = NULL
    xor edx, edx        ; envp = NULL
    mov al, 0x0b        ; sys_execve = 11
    int 0x80
    bash
    # Assemble and verify NO instruction exceeds the 2-byte window:
    python
    python3 - <<'EOF'
    from pwn import *
    context.arch = 'i386'
    context.os = 'linux'
    sc = asm(open('sc.asm').read())
    print('len', len(sc), 'hex', sc.hex())
    EOF
    What didn't work first

    Tried: Use a standard pwntools shellcraft.sh() payload and just send it directly to the service.

    shellcraft.sh() generates a conventional execve shellcode with 5-byte push immediates like 'push 0x68732f6e'. When the loader splices two NOPs after every two of those bytes, the 5-byte instruction's opcode and operand get split across NOP boundaries, producing an illegal or unintended instruction sequence that crashes instead of spawning a shell. Every instruction in the payload must fit within the 2-byte window.

    Tried: Try to use 3-byte instructions like 'mov byte [mem], val' or 'push word imm16' to stay under 4 bytes since only 5-byte pushes seem problematic.

    The NOP splice lands after every two of your bytes, so even a 3-byte instruction gets the third byte separated from its first two by two NOPs, splitting the opcode prefix from its displacement or immediate. Only 1-byte and 2-byte instructions are guaranteed to survive intact; a 3-byte instruction gets cut at byte 2 and the third byte is decoded as a new (usually invalid) opcode after the NOPs.

    Learn more

    A normal execve shellcode does something like push 0x68732f6e (a 5-byte instruction carrying the 4-byte string chunk "n/sh"). Five bytes cannot survive a filter that splices NOPs in every two bytes - the opcode and the immediate get cut apart.

    The fix is to construct the constant in a register byte by byte using only short instructions: xor eax, eax; then loop mov cl, <byte> / add eax, ecx / shl eax, 8 to shift each character into place; then a single push eax. Each of those is one or two bytes, so the inserted NOPs land between complete instructions and do nothing. This is the same idea as "spacing your instructions out" so the filter's insertions are harmless.

    Keep an eye on alignment: because NOPs are inserted after every two of your bytes, an odd-length instruction can shift the splice point into the next instruction. Inserting your own NOP to re-even the alignment (exactly what main does for an odd total length) keeps everything on a clean boundary.

  3. Step 3
    Send the shellcode and get a shell
    Observation
    I noticed the assembled shellcode used only 1- and 2-byte instructions that survived the NOP splice intact, which suggested sending the raw bytes with pwntools to the remote service would trigger execve('/bin/sh') and drop an interactive shell for reading the flag.
    Connect to the service, send the carefully laid-out shellcode, and interact with the resulting shell to read the flag. The loader inserts the NOPs for you, so you send only your real bytes.
    python
    python3 - <<'EOF'
    from pwn import *
    
    context.arch = 'i386'
    p = remote('mercury.picoctf.net', <PORT_FROM_INSTANCE>)
    
    shellcode = asm(open('sc.asm').read())   # the short-instruction execve above
    p.send(shellcode)
    p.interactive()   # then: ls ; cat flag.txt
    EOF

    Expected output

    picoCTF{th4t_w4s_fun_...}
    What didn't work first

    Tried: Use p.sendline() instead of p.send() to send the shellcode.

    sendline() appends a newline byte (0x0a) after your shellcode. That extra byte shifts the total length, potentially making it odd and triggering main's odd-length NOP pad, which inserts an additional 0x90 at the end and can disturb alignment of the final instruction. Use p.send() so the loader receives exactly the bytes you assembled and no more.

    Tried: Connect with nc directly and paste the shellcode as hex or ASCII rather than using pwntools to send raw bytes.

    nc in interactive mode sends characters as text, not raw binary. Pasting hex strings like '31c031db...' sends the ASCII codes for '3', '1', 'c', '0'... not the binary byte 0x31. The loader runs the raw bytes it receives, so you need pwntools (or a similar tool that sends binary) to deliver the assembled opcode bytes intact.

    Learn more

    Once the (NOP-interleaved) shellcode runs execve("/bin/sh", NULL, NULL), you drop into an interactive shell. Use ls and cat flag.txt to read the flag off the server.

Interactive tools
  • Pwntools ForgeGenerate a complete pwntools exploit script from a template: ret2win, shellcode, ret2libc, ROP chain, format string, or blank scaffold. Fill the form, copy or download the .py file. Fully editable before saving.

Flag

Reveal flag

picoCTF{th4t_w4s_fun_...}

The loader inserts NOP bytes between your shellcode bytes (two NOPs after every two of yours), so multi-byte instructions get shredded. Write execve("/bin/sh") using only short (1-2 byte) instructions - build the '/bin/sh' constant byte by byte with mov/add/shl - so the inserted NOPs fall on instruction boundaries and do nothing.

Key takeaway

Shellcode filters that transform input rather than block it require the attacker to understand the exact transform and craft payloads whose semantics survive it. The x86 instruction set encodes instruction length in the opcode itself, so inserting bytes at fixed intervals changes where the CPU decodes instruction boundaries, making long instructions unreliable. The general defense principle is that executable memory regions should be non-writable and writable regions should be non-executable (W^X), because filters applied before execution are an arms race the defender cannot reliably win against a determined attacker who understands the transform.

Related reading

Want more picoCTF 2021 writeups?

Useful tools for Binary Exploitation

What to try next