WinAntiDbg0x300 picoCTF 2024 Solution

Description

1. Step 1Unpack with UPX

   Observation

   strings on the executable shows UPX! markers and sections named UPX0 and UPX1. It is packed, so unpack it before Ghidra can see the real code.

   The binary is packed with UPX. Running strings on the packed file shows UPX! markers and section names like UPX0 and UPX1. Run upx -d WinAntiDbg0x300.exe to unpack it in place, giving you the real code for Ghidra to analyze. Load the PDB file supplied with the challenge during Ghidra import to get function names and variable names for free.

   What didn't work first

   Tried: Loading the packed binary directly into Ghidra and trying to analyze the decompiled output.

   Ghidra sees the UPX stub, not the program. The decompilation looks like a tiny routine copying memory and jumping to a computed address, with none of the real functions in sight. Unpack first.

   Tried: Skipping the PDB import and trying to rename functions manually from context clues.

   Without the PDB, every function is FUN_00401234 and every global is DAT_00403000, and the anti-debug logic disappears among dozens of unlabeled routines. The challenge ships the PDB precisely so you can import it and get the original names back.

   Learn more

   UPX (Ultimate Packer for eXecutables) compresses a binary and prepends a decompression stub. At runtime the stub decompresses the original code into memory and jumps to it. Malware authors use UPX (or custom packers) to shrink payloads and hinder static analysis. The telltale UPX! magic bytes visible via strings make identification trivial. upx -d reverses the packing in place.

   Importing the PDB (Program Database) file during Ghidra analysis provides all the original symbol names: function names, variable names, and type information. This makes reverse engineering dramatically easier. To load a PDB in Ghidra: import the executable, then before analyzing, go to File > Add to Program and browse to the PDB file. Ghidra will warn about an inexact match; accept it.

2. Step 2Understand why you cannot use a debugger

   Observation

   The description warns that the binary fights back and loops forever checking for a debugger. Understand that mechanism before trying to bypass it, and observe with DebugView rather than attaching.

   In Ghidra, examine WinMain and the challenge_create_thread function. The manage_child_process function creates a mutex; if the mutex already exists (meaning a child has been spawned), the child attempts to attach a debugger to the parent and terminate it. The main loop runs forever and the decrypt_flag code is unreachable because an unconditional JMP bypasses it. DebugView captures OutputDebugString calls, letting you see any debug output the program emits without attaching a debugger.

   What didn't work first

   Tried: Attaching x64dbg or OllyDbg to the unpacked binary and trying to step through the anti-debug checks.

   The binary spawns a child that detects the debugger and kills the parent within seconds, so every attempt to pause or step ends the process before you get anywhere. The answer is not to attach at all: DebugView captures the output passively.

   Tried: Running the binary in a terminal and looking for flag output in stdout or a console window.

   OutputDebugString writes to the Windows debug message stream, which never reaches a console or a file, so redirecting stdout captures nothing. DebugView reads that stream, and so would a kernel debugger, which this binary fights.

   Learn more

   DebugView (Dbgview.exe from Microsoft Sysinternals) is a passive monitor for OutputDebugString output. It captures debug messages printed by any process on the system without attaching a debugger, so the anti-debug child-process logic never fires. Run DebugView (32-bit mode for 32-bit binaries) and then run the program normally as administrator.

   The every-5-second output in DebugView ("no debugger was present, exiting successfully") confirms the program is running and its debug output is reachable, but the flag is never printed because the loop never exits to the decrypt_flag call. The solution is to patch the binary so the unconditional JMP is replaced with NOPs, letting execution fall through to the decrypt_flag path.

3. Step 3Patch the binary in Ghidra

   Observation

   In the assembly, challenge_create_thread holds an unconditional JMP that loops forever, so decrypt_flag is never reached. Replace that jump with NOPs and execution falls through to it.

   In Ghidra's assembly view, locate the unconditional JMP at the bottom of the loop in challenge_create_thread. Select those bytes, go to Patch Instruction (or use the byte-level editor), and replace them with NOP instructions. Ghidra binds Patch Instruction to Ctrl+Shift+G; type NOP in the in-place editor once for each byte of the original JMP. Then export the patched binary: File > Export Program > Windows PE. Run the patched binary as administrator with DebugView open and wait up to 5 seconds for the flag to appear.

   What didn't work first

   Tried: Patching the IsDebuggerPresent check or the child-process mutex logic instead of the unconditional JMP in the main loop.

   Neutralizing the anti-debug checks does not help because the loop never exits regardless of whether a debugger is detected - the JMP is unconditional, not conditional. You must patch the JMP that keeps the loop running, not the IsDebuggerPresent comparisons above it.

   Tried: Exporting the patched binary with File > Export Program using the default 'Original File' format instead of 'Windows PE'.

   The Original File export writes back the bytes from the original structure and may leave your patches out. Exporting as Windows PE rebuilds a valid executable with the patched bytes applied, so the NOPs are actually there.

   Learn more

   A NOP (No Operation, opcode 90) is an instruction that does nothing. Replacing a JMP with five NOPs (a JMP instruction is typically 2 to 5 bytes; fill with enough NOPs to cover the original instruction size) causes execution to continue to the next instruction rather than jumping back to the loop head.

   Ghidra's Export Program function writes the patched binary with your in-place edits preserved. The output is a valid PE executable that you can run on Windows. This workflow of: unpack, analyze, patch in Ghidra, export, run with DebugView, is the canonical approach to binaries that actively fight debuggers.

   The challenge_create_thread function also calls compute_hashes functions that modify a global variable. All of those must complete before the flag is decrypted, which is why the flag appears after a delay (about 5 seconds) rather than immediately after the patched JMP is bypassed.

4. Step 4Read the flag from DebugView

   Observation

   The flag goes out through OutputDebugString rather than stdout, and DebugView is already capturing that stream. Run the patched binary with it open.

   After running the patched binary as administrator with DebugView open, wait up to 5 seconds. The program prints 'you got the flag' and then the flag string to the DebugView output pane. Copy picoCTF{...} from there.

   Learn more

   The flag is printed via OutputDebugString, which writes to the Windows debug message stream rather than stdout. Only DebugView (or a proper debugger, which the binary fights) can capture this output. That is why the challenge hints specifically mention DebugView.

   Running as administrator is required because the binary checks for admin privileges as one of its conditions. The patched binary still runs through those checks but now reaches the decrypt_flag path regardless of the debugger presence, so the flag is printed after all the hash computations complete.

Flag

Patching the infinite-loop JMP to NOPs and running the patched binary with DebugView reveals the flag after a short delay.