picoCTF Solutions
PathPart of the Advanced Reverse Engineering path · Step 4 of 5: Keygens and Constraint Solving
picoCTF 2022Reverse EngineeringHard

Keygenme picoCTF 2022 Solution

Published: July 20, 2023

Description

A binary license-key validator builds the correct key at runtime using MD5 hashes and string operations. Rather than reversing every step by hand, run the binary in GDB and read the fully assembled key directly from memory after the program has computed it.

Setup

Download keygenme

Download the binary and make it executable.

Open in Ghidra to understand the structure, then run in GDB on the picoCTF web shell to extract the key dynamically.

bash
wget https://artifacts.picoctf.net/c/244/keygenme && chmod +x keygenme
bash
gdb keygenme

Solution

Want to try it yourself first?

The guided walkthrough reveals hints one step at a time.

Walk me through it
  1. Step 1
    Find the key assembly point in Ghidra
    Observation
    I noticed the binary is a license-key validator with no source code available, which suggested static analysis in Ghidra was needed to map out where the key is assembled before switching to dynamic extraction in GDB.
    Open the binary in Ghidra, run auto-analysis, and browse to the license-check function. Look for the call to strlen (which fires once the key string is complete). That call site is a good breakpoint because the full key lives in memory at rbp-0x30 at that moment.
    Learn more

    Ghidra decompiles the key-check function and shows the sequence of MD5 calls and sprintf operations that assemble the expected license key. The decompiler output reveals that the assembled string is stored at a local variable corresponding to rbp-0x30 on the stack frame.

    You do not need to understand every MD5 computation. The goal is to find the address where the string is complete so you can break there in a debugger and read the value directly from memory.

  2. Step 2
    Break after key assembly in GDB and read the flag
    Observation
    I noticed Ghidra revealed that the key is stored at rbp-0x30 right after the strlen call, which suggested setting a GDB breakpoint at that exact address to read the fully assembled key from the stack without needing to invert any MD5 computation.
    Start the binary under GDB. It loads into libc_start_main initially with no main symbol defined. Use info address main or step through entry to find main's address, then set a breakpoint just after the strlen call where the key is complete. Examine the string at $rbp-0x30 to read the flag.
    bash
    gdb keygenme
    bash
    # GDB session:
    bash
    run
    bash
    # After it prints the prompt and pauses, break on the check function:
    bash
    info functions
    bash
    # Set breakpoint at the strlen call site (address from Ghidra):
    bash
    break *0x<ADDRESS_FROM_GHIDRA>
    bash
    continue
    bash
    # Enter any license key when prompted (it will not matter)
    bash
    x/s $rbp-0x30
    What didn't work first

    Tried: Try to reverse the MD5 chain manually by cracking the hash values found in Ghidra

    MD5 is a one-way function, so cracking it requires a preimage attack or a dictionary/rainbow-table match. The key is assembled from MD5 digests of internal constants that are not dictionary words, so hashcat and online crackers return no results. The debugger approach sidesteps this entirely by reading the already-computed answer from the stack buffer before the comparison runs.

    Tried: Break at the strcmp or strncmp call instead of the strlen call site

    The comparison function receives two arguments: the user-supplied input and the expected key. If you break at strcmp, the key pointer is in rsi (for both strcmp and strncmp), not at rbp-0x30, so x/s $rbp-0x30 may print garbage or an earlier stack remnant. The strlen call fires right after the key is fully assembled and stored at rbp-0x30, making that address reliable. At the comparison site, use x/s $rsi instead.

    Learn more

    The binary calculates the correct license key at runtime and stores it in a stack buffer. GDB's x/s $rbp-0x30 command dereferences the address and prints it as a null-terminated string. Because the key is built before any comparison with user input, you can provide a dummy key to get past the prompt, hit the breakpoint, and read the real key.

    This dynamic approach works for any validator that builds the expected value at runtime - even if the algorithm is a one-way hash chain that cannot be inverted analytically. As long as the program computes the answer to compare against, a debugger can intercept it.

    On the picoCTF web shell, GDB is pre-installed. Run gdb ./keygenme, then use run to start. If the binary has no main symbol, break on __libc_start_main first, step through to find main, then set your real breakpoint.

  3. Step 3
    Submit the key
    Observation
    I noticed the GDB session printed the complete picoCTF flag string at the expected stack address, which confirmed the extracted value was the correct license key and could be submitted directly to the binary.
    Run the binary normally and paste the key read from GDB. The validator accepts it and prints the flag.
    bash
    ./keygenme
    bash
    # Enter the key from GDB output: picoCTF{br1ng_y0ur_0wn_k3y_...}

    Expected output

    picoCTF{br1ng_y0ur_0wn_k3y_...}
    Learn more

    The key is the flag itself formatted as picoCTF{bring_your_own_key_...}. After entering it, the binary prints a success message confirming the flag.

    The key insight from this challenge: dynamic analysis with GDB often bypasses the need to fully reverse-engineer a complex algorithm. If the program needs to know the right answer to compare against, the right answer is somewhere in memory right before the comparison happens.

Flag

Reveal flag

picoCTF{br1ng_y0ur_0wn_k3y_...}

Use GDB to break after the key-assembly code and read the expected key directly from the stack at rbp-0x30. No MD5 reversal needed.

Key takeaway

Dynamic analysis with a debugger sidesteps one-way algorithms by intercepting the computed answer in memory at the moment of comparison, before any hash or transform is applied. Any license validator or authentication check that builds the expected value at runtime is vulnerable to this technique regardless of how complex the algorithm is. The same approach applies to malware unpacking (intercepting the decrypted payload after the unpacker runs), anti-cheat bypass, and firmware license checks where the derived key surfaces in a register or stack buffer just before the comparison instruction.

Related reading

Want more picoCTF 2022 writeups?

Browse picoCTF 2022Full library

Useful tools for Reverse Engineering

What to try next