unpackme picoCTF 2022 Solution

Description

1. Step 1Unpack the UPX binary

   Observation

   The challenge name says UPX, and strings returns mostly garbage alongside the four-byte UPX! tag. The binary is UPX-compressed, so run upx -d to restore the original ELF before analyzing anything.

   Run upx -d unpackme-upx to decompress the binary in-place (or use -o to write a new file). The decompressed ELF can then be analyzed normally.

   bash

   upx -d unpackme-upx -o unpackme-unpacked

   Copied!

   bash

   ls -lh unpackme-upx unpackme-unpacked

   Copied!

   What didn't work first

   Tried: Run strings on the packed binary hoping to find the flag or a readable password.

   The packed binary's code segment is compressed, so strings returns noise and a few UPX stub artifacts. Anything interesting, the scanf format specifier and the surrounding text, only appears once upx -d has decompressed the ELF. Running strings on the packed file goes nowhere.

   Tried: Try upx -d without the -o flag, overwriting the original binary in-place, then discover the original packed file is gone.

   upx -d without -o overwrites the input with the decompressed version, which is fine while you still have the original download. Want to compare file sizes or repeat the experiment and you are downloading again. Pass -o to keep the original beside the decompressed copy.

   Learn more

   UPX (Ultimate Packer for eXecutables) is a free, open-source packer that compresses executable files. At runtime, the UPX stub decompresses the original code into memory and jumps to the original entry point. Packed binaries are smaller on disk but unpack to their full size in memory.

   Malware commonly uses UPX (or custom packers) to evade antivirus signature scanning - the raw bytes of the packed file don't match the signatures for the malicious code inside. Analysts unpack first, then scan or reverse engineer the inner binary.

   UPX stores a magic header (UPX!) at the end of compressed segments. The -d flag decompresses; -l lists packing info. Running strings on a packed binary shows mostly garbage; after unpacking, meaningful strings like function names and the password comparison become visible.

2. Step 2Find the integer check in Ghidra

   Observation

   The unpacked binary asks for a favourite number rather than a password. The secret is a numeric constant sitting in a cmp instruction, which Ghidra's decompiler will show directly in the pseudocode of main.

   Open the unpacked binary in Ghidra. Navigate to main() and look for the scanf call reading an integer, followed by a comparison of the local variable against the constant 0xb83cb. The binary asks 'What's my favorite number?' and expects a decimal integer, not a string.

   What didn't work first

   Tried: Search for strcmp or strncmp calls in Ghidra expecting a hardcoded string password.

   The binary reads an integer with scanf and compares it with a cmp instruction, never a string comparison, so the Functions window and cross-references show no strcmp at all. Searching string references turns up the prompt and the flag format and no password. The secret is a numeric constant, visible only as a hex literal in the cmp operand.

   Tried: Run ltrace ./unpackme-unpacked and enter a guess, expecting ltrace to reveal what the binary compares against.

   ltrace intercepts dynamic library calls, and this comparison is a bare cmp instruction inside main rather than a library function. The output shows the scanf call and never a strcmp or memcmp, so nothing leaks. Break on the cmp in GDB, or read it statically in Ghidra.

   Learn more

   Ghidra is NSA's free reverse engineering framework. After importing the binary and running auto-analysis, use the Symbol Tree panel to navigate directly to main. Ghidra's decompiler (open via Window > Decompiler, or the docked Decompile panel) converts the disassembly into readable C pseudocode.

   Finding main in a stripped binary. UPX-unpacked binaries are usually stripped (no main symbol). The standard trick is to find the call to __libc_start_main: the very first argument to it is a function pointer to main. Two ways:

   # objdump way: print the disassembly around every __libc_start_main call.
   objdump -d unpackme-unpacked | grep -B5 '__libc_start_main'
   
   # Look for the immediately-preceding instruction that loads RDI (System V x86-64),
   # e.g. "lea rdi, [rip+0xNNN]" -> RIP-relative address of main.
   
   # In Ghidra: open the entry function (usually _start), follow the first argument
   # loaded into RDI before the call __libc_start_main, double-click the address.

   In Ghidra's decompiler view of main, you will see a pattern like scanf("%d", &local_44) followed by if (local_44 == 0xb83cb). This is a direct integer comparison, not a string comparison. Convert the hex constant to decimal: 0xb83cb = 754635. That is the number you enter.

   Integer vs. string comparison. The disassembly uses a cmp instruction directly on the register holding the scanned integer, rather than calling strcmp. There is no hardcoded string password in this binary. The distinction matters for dynamic analysis too: ltrace will not show a strcmp call here because none exists.

3. Step 3Run the binary with the correct number

   Observation

   Ghidra shows the comparison against 0xb83cb, with scanf reading a decimal integer. Convert that constant to decimal, 754635, type it at the prompt, and the check passes.

   When prompted 'What's my favorite number?', enter 754635 (the decimal form of 0xb83cb). The binary compares your input to that constant and, on a match, prints the flag.

   bash

   ./unpackme-unpacked

   Copied!

   bash

   # When prompted, enter: 754635

   Copied!

   Expected output

   What's my favorite number? 754635
   picoCTF{up><_m3_f7w_...}

   What didn't work first

   Tried: Enter 0xb83cb (the hex literal directly) at the prompt instead of the decimal value 754635.

   The %d format reads a decimal integer, so typing the hex form is not understood: scanf takes the leading zero, stops at the x, and hands back 0, which fails the comparison. Convert the constant to decimal before entering it.

   Tried: Run the still-packed unpackme-upx binary directly instead of the unpacked copy, then enter 754635.

   The packed binary decompresses itself at runtime and accepts the same number, so the answer does not change. But running it without unpacking leaves nothing to inspect statically: Ghidra analyzes the UPX stub rather than the real code. Unpack first so static analysis is possible at all.

   Learn more

   Once you have the constant from static analysis, simply run the binary normally and type 754635 when prompted. The cmp instruction succeeds and the flag is printed to stdout.

   An alternative to static analysis is dynamic analysis with GDB: set a breakpoint at the comparison instruction, run the binary, and inspect the register or stack value being compared. Because the comparison is a direct integer cmp rather than a library call, ltrace will not capture it. Use strace or GDB instead for dynamic approaches.

   Hex-to-decimal conversion. The constant 0xb83cb can be converted quickly with Python: python3 -c "print(0xb83cb)" prints 754635. Always convert hex constants to decimal before entering them at a decimal scanf prompt, or the comparison will fail.

   strace (system call trace) and ltrace (library call trace) are essential dynamic analysis tools for understanding what a binary does without fully reversing it. They are commonly combined with GDB for a complete picture, especially when dealing with direct integer comparisons that bypass library functions.

Flag

Unpack with `upx -d`, find the integer constant 0xb83cb (754635) in Ghidra's decompiled main(), then enter it when the binary asks 'What\'s my favorite number?'.