Skip to main content
picoCTF Solutions
PathPart of the Advanced Reverse Engineering path · Step 4 of 5: Keygens and Constraint Solving
picoCTF 2022Reverse EngineeringHard

Keygenme picoCTF 2022 Solution

Reverse engineer a binary that validates license keys using a complex algorithmic constraint.

Published: July 20, 2023Updated: August 13, 2026

Description

A binary license-key validator builds the correct key at runtime using MD5 hashes and string operations. Rather than reversing every step by hand, run the binary in GDB and read the fully assembled key directly from memory after the program has computed it.

Setup

Download keygenme

Download the binary and make it executable.

Open in Ghidra to understand the structure, then run in GDB on the picoCTF web shell to extract the key dynamically.

bash
wget https://artifacts.picoctf.net/c/244/keygenme && chmod +x keygenme
bash
gdb keygenme

Solution

Want to try it yourself first?

The guided walkthrough reveals hints one step at a time.

Walk me through it
  1. Step 1Find the key assembly point in Ghidra
    Observation
    The binary is a license-key validator with no source available. Map where the key is assembled with static analysis in Ghidra, then switch to GDB to extract it.
    Open the binary in Ghidra, run auto-analysis, and browse to the license-check function. Look for the call to strlen (which fires once the key string is complete). That call site is a good breakpoint because the full key lives in memory at rbp-0x30 at that moment.
    Learn more

    Ghidra decompiles the key-check function and shows the sequence of MD5 calls and sprintf operations that assemble the expected license key. The decompiler output reveals that the assembled string is stored at a local variable corresponding to rbp-0x30 on the stack frame.

    You do not need to understand every MD5 computation. The goal is to find the address where the string is complete so you can break there in a debugger and read the value directly from memory.

  2. Step 2Break after key assembly in GDB and read the flag
    Observation
    Ghidra shows the key stored at rbp-0x30 immediately after the strlen call. Break there in GDB and read the fully assembled key off the stack, with no MD5 to invert.
    Start the binary under GDB. It loads into libc_start_main initially with no main symbol defined. Use info address main or step through entry to find main's address, then set a breakpoint just after the strlen call where the key is complete. Examine the string at $rbp-0x30 to read the flag.
    bash
    gdb keygenme
    bash
    # GDB session:
    bash
    run
    bash
    # After it prints the prompt and pauses, break on the check function:
    bash
    info functions
    bash
    # Set breakpoint at the strlen call site (address from Ghidra):
    bash
    break *0x<ADDRESS_FROM_GHIDRA>
    bash
    continue
    bash
    # Enter any license key when prompted (it will not matter)
    bash
    x/s $rbp-0x30
    What didn't work first

    Tried: Try to reverse the MD5 chain manually by cracking the hash values found in Ghidra

    MD5 is one-way, so cracking it needs a preimage attack or a dictionary match. This key is assembled from digests of internal constants that are not dictionary words, so hashcat and online crackers return nothing. The debugger route skips all of it, reading the already-computed answer off the stack before the comparison runs.

    Tried: Break at the strcmp or strncmp call instead of the strlen call site

    The comparison takes two arguments, your input and the expected key. Break at strcmp and the key pointer is in rsi rather than at rbp-0x30, so reading that stack address prints garbage or an earlier remnant. The strlen call fires right after the key is assembled and stored there, which makes that address reliable. At the comparison site, read rsi instead.

    Learn more

    The binary calculates the correct license key at runtime and stores it in a stack buffer. GDB's x/s $rbp-0x30 command dereferences the address and prints it as a null-terminated string. Because the key is built before any comparison with user input, you can provide a dummy key to get past the prompt, hit the breakpoint, and read the real key.

    This dynamic approach works for any validator that builds the expected value at runtime - even if the algorithm is a one-way hash chain that cannot be inverted analytically. As long as the program computes the answer to compare against, a debugger can intercept it.

    On the picoCTF web shell, GDB is pre-installed. Run gdb ./keygenme, then use run to start. If the binary has no main symbol, break on __libc_start_main first, step through to find main, then set your real breakpoint.

  3. Step 3Submit the key
    Observation
    GDB prints a complete picoCTF string at the expected stack address, which confirms the extracted value is the license key and can go straight back into the binary.
    Run the binary normally and paste the key read from GDB. The validator accepts it and prints the flag.
    bash
    ./keygenme
    bash
    # Enter the key from GDB output: picoCTF{br1ng_y0ur_0wn_k3y_...}

    Expected output

    picoCTF{br1ng_y0ur_0wn_k3y_...}
    Learn more

    The key is the flag itself formatted as picoCTF{bring_your_own_key_...}. After entering it, the binary prints a success message confirming the flag.

    The key insight from this challenge: dynamic analysis with GDB often bypasses the need to fully reverse-engineer a complex algorithm. If the program needs to know the right answer to compare against, the right answer is somewhere in memory right before the comparison happens.

Interactive tools
  • Strings ExtractorPull printable text from any binary, library, or image. ASCII and UTF-16 detection, configurable minimum length, flag-like highlight, no command line needed.
  • Hex ViewerView text or raw hex bytes as a xxd-style hex dump with byte offset, hex columns, and ASCII sidebar. Highlights printable characters and null bytes.

Flag

Reveal flag

picoCTF{br1ng_y0ur_0wn_k3y_...}

Use GDB to break after the key-assembly code and read the expected key directly from the stack at rbp-0x30. No MD5 reversal needed.

Key takeaway

A debugger sidesteps one-way algorithms by catching the computed answer in memory at the moment of comparison, before any hash or transform matters. Any license validator or authentication check that builds its expected value at runtime falls to this, however complex the algorithm. The same move unpacks malware by intercepting the decrypted payload, bypasses anti-cheat, and defeats firmware license checks, wherever the derived key surfaces in a register or stack buffer just before the comparison.

Related reading

Useful tools for Reverse Engineering

Where to go next

Try these next