Investigative Reversing 2 picoCTF 2019 Solution

Published: April 2, 2026

Description

Two files: a 64-bit ELF binary called mystery and a bitmap image called encoded.bmp. The binary embedded the flag into the image using LSB steganography with an arithmetic twist. Reverse-engineer the encoder to write a decoder that recovers the flag.

Download both files and make the binary executable.

bash
wget <url>/mystery
bash
wget <url>/encoded.bmp
bash
chmod +x mystery

Solution

Want to try it yourself first?

The guided walkthrough reveals hints one step at a time.

Walk me through it
  1. Step 1
    Reverse-engineer the binary in Ghidra
    Observation
    I noticed the challenge provided a compiled ELF binary alongside the encoded image, which suggested the binary contained the encoding logic and that decompiling it in Ghidra was the only way to discover the exact arithmetic transform and byte offset needed to write a correct decoder.
    Open mystery in Ghidra and decompile it. The binary reads a 50-character flag from flag.txt, subtracts 5 from each character's ASCII value, then encodes the result bit-by-bit into the least significant bit (LSB) of consecutive bytes in an output BMP. The first 2000 bytes (offset 0x7d0) of the BMP are copied unchanged; the encoded region starts immediately after and spans 400 bytes (50 characters x 8 bits each).
    bash
    ghidra mystery &
    What didn't work first

    Tried: Running strings or ltrace on the binary to find the encoding logic without Ghidra.

    strings reveals the filenames (flag.txt, encoded.bmp) but not the arithmetic transform or byte offset. ltrace shows libc calls like fread and fwrite but hides the per-bit manipulation inside a custom loop. Only decompiling in Ghidra exposes the -5 subtraction and the exact LSB encoding expression.

    Tried: Using steghide or zsteg to extract the hidden data automatically.

    steghide expects a passphrase-protected payload embedded with its own header format, so it reports 'steghide: the bmp-file does not contain any embedded data.' zsteg works on PNG and may misidentify the BMP channel layout. Neither tool knows about the custom -5 arithmetic shift, so even if they extracted raw bits the output would be offset by 5 for every character.

    Learn more

    LSB steganography works by replacing just the lowest bit of each carrier byte with one bit of secret data. Changing a byte from 0xe8 to 0xe9 is a difference of 1, which is invisible to the eye but detectable by reading those bits back. The encoder here uses the C expression (carrier_byte & 0xfe) | (flag_bit & 1), which zeroes the LSB and then OR-s in the new bit.

    The -5 arithmetic shift is a simple obfuscation layer. If the flag character is 'p' (ASCII 112), the encoder embeds 107 instead. The decoder must add 5 back to recover the original character.

  2. Step 2
    Extract LSBs from the encoded region and decode the flag
    Observation
    I noticed from the Ghidra decompilation that the encoder seeked to offset 0x7d0 (2000) in the BMP, stored flag bits in LSB order with a -5 subtraction applied, which meant the decoder needed to skip the same 2000-byte header, extract LSBs in LSB-first bit order, and add 5 back to recover the original characters.
    Seek to byte offset 2000 in encoded.bmp, then read 400 bytes. Extract the LSB of each byte in order. Every 8 consecutive bits form one character. After reassembling the 8-bit value, add 5 to reverse the encoder's subtraction and convert to a character.
    python
    python3 << 'EOF'
    with open('encoded.bmp', 'rb') as f:
        f.seek(2000)
        data = f.read(50 * 8)  # 50 chars, 8 bits each
    
    bits = [byte & 1 for byte in data]
    
    flag = ''
    for i in range(50):
        byte_bits = bits[i * 8:(i + 1) * 8]
        # bits were stored LSB-first (little-endian bit order)
        value = 0
        for j, bit in enumerate(byte_bits):
            value |= bit << j
        flag += chr(value + 5)  # reverse the encoder's -5 shift
    
    print(flag)
    EOF
    What didn't work first

    Tried: Reading from offset 0 instead of 2000, treating the entire BMP including the header as the encoded region.

    The first 2000 bytes of the BMP are copied verbatim by the encoder, so extracting LSBs from them produces noise from the BMP header and unmodified pixel data. The reassembled characters will not be printable ASCII. The Ghidra decompilation shows the skip explicitly: the encoder seeks to 0x7d0 (decimal 2000) before beginning the bit-writing loop.

    Tried: Reconstructing each byte MSB-first (bit 7 down to bit 0) instead of LSB-first.

    If you loop j from 7 down to 0 and write value |= bit << j, you reverse the bit order relative to what the encoder stored. Every character comes out as a different wrong value, typically non-printable. The encoder stores bit 0 of the character first (j=0 in the inner loop), so the decoder must assign the first extracted bit to position 0 of the output byte.

    Learn more

    The bit order matters. Ghidra will show the encoder looping from bit position 0 to 7 (LSB to MSB), so the decoder must reconstruct the byte using the same order: bit 0 of the extracted sequence is bit 0 of the character value. The expression value |= bit << j does exactly that.

    After adding 5 back, every character should fall in the printable ASCII range. If you see garbage, double-check the byte order and the offset (some builds use a slightly different offset based on BMP header size).

Interactive tools
  • StegallDrop any file and Stegall runs every applicable steg technique in parallel: LSB sweeps, bit planes, spectrograms, polyglot carving, metadata, whitespace decode, and a 6-layer base/ROT/XOR/zlib cascade. Recursively unpacks results and surfaces flag matches.
  • Hex ViewerView text or raw hex bytes as a xxd-style hex dump with byte offset, hex columns, and ASCII sidebar. Highlights printable characters and null bytes.
  • Strings ExtractorPull printable text from any binary, library, or image. ASCII and UTF-16 detection, configurable minimum length, flag-like highlight, no command line needed.

Flag

Reveal flag

picoCTF{n3xt_0n3_...}

The flag hash suffix is instance-specific and will differ between runs of the challenge.

Key takeaway

LSB steganography hides data by replacing the least significant bit of each pixel or sample byte, a change too small to perceive visually or audibly but perfectly recoverable by reading those bits back in order. When combined with a simple arithmetic transform on the secret values before embedding, the technique adds a lightweight obfuscation layer that requires understanding the encoder to undo. The same primitive is used in real-world digital watermarking, covert data exfiltration over image-sharing platforms, and tools like steghide and zsteg that automate both hiding and detection.

Related reading

Want more picoCTF 2019 writeups?

Useful tools for Forensics

What to try next