Investigative Reversing 3 picoCTF 2019 Solution

Description

1. Step 1

   Decompile the binary in Ghidra

   Observation

   I noticed the challenge provides a compiled binary alongside an encoded BMP but no source code, which suggested I needed to decompile mystery in Ghidra to discover the exact start offset and stride pattern the encoder used before attempting any extraction.

   Load mystery into Ghidra and inspect the decompiled main. The binary opens flag.txt, original.bmp, and encoded.bmp. It reads the flag one byte at a time and encodes each bit into the LSB of a corresponding byte in encoded.bmp, but with a twist: after every 8 encoded bytes it writes one unmodified byte from the original image. That 8-encoded + 1-passthrough pattern repeats 100 times starting at file offset 0x2d3 (decimal 723).

   bash

   ghidra mystery &

   Copied!

   What didn't work first

   Tried: Run strings or strace on mystery to infer the encoding scheme without Ghidra.

   strings shows the filenames (flag.txt, original.bmp, encoded.bmp) but reveals nothing about the 9-byte cycle or the start offset 0x2d3. strace shows file open and read calls but not the loop arithmetic. Without the decompiled loop body you will guess wrong parameters and extract garbage - the only reliable source is the decompiler output.

   Tried: Assume the encoding matches the earlier Investigative Reversing challenges and reuse a previous extractor script directly.

   Earlier challenges in this series use a simpler stride with no passthrough bytes and a different start offset. Applying that script to this file produces a garbled string because every 9th byte is consumed as data instead of being skipped. You must read this binary's decompiled loop to learn the 8-encoded + 1-passthrough cycle before writing any extractor.

   Learn more

   LSB steganography works by replacing the lowest-order bit of each image byte with one bit of hidden data. A 1-bit change shifts the byte value by at most 1, which is invisible to the eye but detectable by reading that single bit back. The 9-byte cycle here (8 encoded + 1 passthrough) is a deliberate complexity layered on top of basic LSB: the passthrough byte acts as padding that the decoder must skip.

2. Step 2

   Confirm the encoding pattern with xxd

   Observation

   I noticed Ghidra's decompiled loop pointed to offset 0x2d3 as the encoding start and a 9-byte cycle, which suggested verifying those parameters directly against the raw bytes with xxd before coding the extractor to avoid wasting time on a wrong implementation.

   Before writing the extractor, verify the offset. BMP pixel data in this file begins at byte 54 (the standard BMP header size), but the flag encoding starts much later at 0x2d3. Use xxd to inspect that region and confirm the 9-byte cycle boundary by comparing encoded.bmp to the unmodified original.

   bash

   xxd encoded.bmp | grep -A 5 '000002d0'

   Copied!

   Expected output

   000002d0: 5050 4f4f 5150 5052 50                   PPOOPQPPR.

   What didn't work first

   Tried: Start inspecting LSB differences from byte 54, the standard BMP pixel data offset, instead of 0x2d3.

   The standard BMP header ends at byte 54, but the encoder deliberately skips the first 0x2d3 bytes before writing flag bits. Bytes 54 through 0x2d2 are unmodified original image data, so diffing that region against the original shows no differences and gives the false impression that the encoding has not started yet. The xxd grep at the correct offset 0x2d0 is needed to land on the actual encoding boundary.

   Tried: Use a visual diff tool or steghide to detect the modified region rather than xxd.

   Visual diff tools compare pixel colors and will not highlight the 1-bit LSB changes because the color shift is at most 1 intensity unit - well below perception threshold. steghide expects a passphrase and uses its own embedding algorithm, so it will fail to extract anything meaningful from a binary that rolled its own encoding scheme. xxd with a specific offset is the right approach because it shows raw byte values you can compare manually against the 9-byte cycle boundary.

   Learn more

   BMP files store pixels row by row, starting after a fixed header. The encoding in this challenge deliberately skips the first 0x2d3 bytes so the image header and some initial pixel data remain clean, making a quick visual comparison harder to spot.

3. Step 3

   Write the extractor

   Observation

   I noticed Ghidra confirmed the 8-encoded + 1-passthrough cycle starting at 0x2d3 and xxd validated the boundary, which meant I had all the parameters needed to write a Python extractor that reads 9 bytes per flag character, collects the LSB of the first 8, and skips the 9th.

   Open encoded.bmp as raw bytes, seek to offset 0x2d3, then loop 50 times (one iteration per flag character). For each character, read 9 bytes: collect the LSB of each of the first 8 bytes to build a bit string (LSB-first, little-endian), then skip the 9th passthrough byte. Convert the 8-bit string to a character and append it to the flag.

   python

   python3 << 'EOF'
   with open('encoded.bmp', 'rb') as f:
       f.seek(0x2d3)
       data = f.read(50 * 9)  # 50 characters, 9 bytes each
   
   flag = ''
   for i in range(50):
       bits = ''
       for j in range(8):          # 8 LSB-encoded bytes per character
           byte = data[i * 9 + j]
           bits += str(byte & 1)   # collect LSB, LSB-first order
       char_val = int(bits[::-1], 2)  # reverse for big-endian byte value
       if char_val == 0:
           break
       flag += chr(char_val)
   
   print(flag)
   EOF

   Copied!

   Expected output

   picoCTF{4n0th3r_L5b_pr0bl3m_...}

   What didn't work first

   Tried: Collect LSB bits in MSB-first order and convert directly without reversing the bit string.

   The binary writes the least-significant bit of each flag byte first into successive LSB slots, so the raw bit string is LSB-first. If you call int(bits, 2) without reversing, bit 0 (the LSB of the character) is treated as the most significant position and you get the wrong character. The [::-1] reversal corrects the ordering before conversion; you can verify it is correct because the first decoded character should be 'p' (0x70 = 0111 0000).

   Tried: Read all 50 * 9 = 450 bytes and then iterate over them with a stride of 8 instead of 9, ignoring the passthrough bytes entirely.

   Striding by 8 instead of 9 means by the 9th character you are consuming what is actually the passthrough byte of the 8th character as a data byte, and all subsequent characters shift one position forward in the buffer. The output will look plausible for the first few characters (picoCTF{...) but corrupts from character 9 onward. The inner loop must explicitly consume 8 data bytes then skip 1 passthrough byte per iteration to stay aligned.

   Learn more

   Bit-ordering matters: the binary encodes the least-significant bit of each flag byte into the first LSB slot it writes, so the collected bit string is LSB-first and must be reversed before converting to an integer with int(bits[::-1], 2). Always verify bit order by checking known bytes - the 'p' in 'picoCTF' is 0x70 (0111 0000), so the first few extracted bits should spell that out once correctly ordered.

Flag

The hash suffix is per-instance and will differ from the value above. Run the extractor against your own encoded.bmp to get the correct flag.