File types picoCTF 2022 Solution

Description

1. Step 1

   Peel each layer

   Observation

   I noticed that running sh Flag.pdf produced a file named flag with no extension and binary content, which suggested the archive was layered and that the file command on the extracted output (rather than the name or extension) was the only reliable way to determine each successive format and choose the correct extractor.

   After each extraction, run file flag to identify the next compression/container type and use the matching extractor.

   bash

   # Peel one layer at a time. After EACH extraction, re-run 'file flag'
   # to see the next layer, then use the matching extractor below.
   # The trick: rename 'flag' to the extension the tool expects; the
   # decompressor strips it and writes 'flag' back, ready for the next pass.
   file flag                                   # identify the current layer, then run ONE of:
   
   ar x flag && rm -f flag                     # "ar archive"   -> extracts the inner member
   cpio -idu < flag                            # "cpio archive" -> extracts the inner file
   mv flag flag.bz2  && bunzip2 flag.bz2       # "bzip2 compressed"
   mv flag flag.gz   && gunzip  flag.gz        # "gzip compressed"
   mv flag flag.xz   && unxz    flag.xz        # "XZ compressed"
   mv flag flag.lzma && unlzma  flag.lzma      # "LZMA compressed"
   mv flag flag.lz   && lzip -d flag.lz        # "lzip compressed"
   mv flag flag.lz4  && unlz4 -f flag.lz4 flag # "LZ4 compressed"
   mv flag flag.lzo  && lzop -d flag.lzo       # "lzop compressed"
   
   # Repeat until 'file flag' reports ASCII text, then hex-decode (next step).

   Copied!

   What didn't work first

   Tried: Trying to open Flag.pdf in a PDF viewer or running pdftotext Flag.pdf expecting readable content

   The file has a .pdf extension but its magic bytes identify it as a shell archive (shar), not a PDF. pdftotext will either error out or produce garbage because there is no PDF structure inside. The correct approach is sh Flag.pdf which executes the embedded shell commands and extracts the first real file.

   Tried: Skipping file flag between extractions and guessing the next extractor from the filename or the previous layer

   Each extracted file is renamed flag with no extension, and each layer uses a different compression format. Guessing (e.g. running bunzip2 on what is actually an xz archive) produces a 'not a bzip2 file' error and may corrupt the file in place. Running file flag after every single extraction step is the only reliable way to identify the next format from its magic bytes.

   Learn more

   The file command reads magic bytes - the first few bytes of the file - to identify the format regardless of extension. file works because every container has a unique signature in its header. Quick reference:

   Magic bytes (first 2-6 bytes):
   
     gzip       1f 8b 08
     bzip2      42 5a 68            ("BZh")
     xz         fd 37 7a 58 5a 00   ("\xfd7zXZ" + "\0")
     lzma       5d 00 00            (no good universal magic)
     lz4        04 22 4d 18
     cpio       30 37 30 37 30      (ASCII "07070")
     ar         21 3c 61 72 63 68   ("!<arch>")
     zip / jar  50 4b 03 04         ("PK\x03\x04")
     PNG        89 50 4e 47         ("\x89PNG")
     ELF        7f 45 4c 46         ("\x7fELF")

   Why ship the chain inside a shell archive (shar) wearing a PDF extension? Layered misdirection. PDF makes you expect a binary you'd open in a viewer, not a script you'd execute. shar predates tar - it's a self-extracting shell script that recreates files via inline commands. The format is harmless in itself; the trick is that file Flag.pdf identifies it as a shell script regardless of extension.

   Entropy heuristic. Compressed/encrypted data has high entropy (~7.99/8.0 bits per byte); plaintext is around 4-5. ent flag or python3 -c "import collections, math; b=open('flag','rb').read(); print(-sum((c/len(b))*math.log2(c/len(b)) for c in collections.Counter(b).values()))" gives you a quick check: if entropy stays high, you're still wrapped; if it drops, you've hit text or hex.

   More CLI recipes for this kind of file archaeology in Linux CLI for CTF.

2. Step 2

   Decode the hex

   Observation

   I noticed that after all compression layers were removed, file flag reported ASCII text and head flag showed a string of hex digit pairs rather than a readable flag, which suggested the final encoding step was a plain hex dump that xxd -r -p could reverse into the actual flag bytes.

   The final file is ASCII hex; xxd -r -p converts it back to readable bytes. Verify the output looks like a flag before submitting.

   bash

   head -c 64 flag

   Copied!

   bash

   xxd -r -p flag

   Copied!

   bash

   xxd -r -p flag | head -c 200

   Copied!

   Expected output

   picoCTF{f1len@m3_m@n1pul@t10n_f0r_0b2cur17y_3c7...}

   What didn't work first

   Tried: Running xxd flag instead of xxd -r -p flag and reading the hex output as if it were the flag

   xxd flag without flags dumps the file contents as hex, which just shows you the ASCII hex digits displayed as hex again - a double-encoding. The -r flag is required to reverse the process (convert hex text back to bytes) and -p tells xxd the input is plain hex with no offset columns. Without both flags, you get the opposite of what you need.

   Tried: Running xxd -r -p flag when file flag still reports a compressed format rather than ASCII text

   If an extraction layer was missed, the file still contains binary compressed data, not ASCII hex digits. Passing binary data to xxd -r -p produces garbled output because xxd tries to parse non-hex bytes as hex pairs. The fix is to go back and run file flag - if it reports anything other than ASCII text, there is at least one more extraction step needed before hex-decoding.

   Learn more

   Hex round-trip example. If cat flag shows 706963 6f4354 467b66 316c65..., then xxd -r -p flag emits the bytes p i c o C T F { f 1 l e .... -r reverses (read hex, write bytes), -p selects "plain" format (just the hex digits, no offsets, no ASCII sidebar). That's the same format Python's bytes.hex() produces, so it round-trips cleanly.

   Sanity-check the result: it should start with picoCTF{. If it's still binary garbage, you missed an extraction layer. Run file on the "hex" input first; if file says it's still a compressed format, you stopped peeling too early.

   The general lesson: in forensics, extension is a hint and magic bytes are the truth. file and the magic-byte table above let you correctly identify any container regardless of how an attacker labelled it. Hex-dump fundamentals in Hex Dumps for CTF.

Flag

Automating the extraction loop with `while file flag | grep ...` can save time on nested compression challenges.