Description
A new password authentication program that even shows you the password you entered in the database. Isn't that cool? Download system.out and recover what hash value it is actually checking.
Setup
chmod +x system.out./system.outSolution
Want to try it yourself first?
The guided walkthrough reveals hints one step at a time.
Step 1Understand the program in Ghidra
ObservationThe challenge gives a native ELF and asks for the hash value it checks against. Decompile it in Ghidra for the hash algorithm, the obfuscation bytes, and the comparison in main().Load system.out into Ghidra and analyze it. Rename variables to make it readable. The program: (1) calloc's a 90-byte heap buffer, (2) writes 13 embedded bytes XOR'd with 0xAA into that buffer at offset 60, (3) asks you to set a password and reads up to 50 chars with fgets, (4) asks how many bytes in length your password is, (5) prints that many bytes of the buffer back as decimal numbers, (6) asks for your hash value and parses it with strtoul. It calls make_secret(), which decodes the same embedded bytes into a fresh buffer, terminates it at index 12, and hashes it. The hash is djb2: total starts at 0x1505 (5381) and for each byte total = (total * 33) + byte_value, kept in a 64-bit unsigned long. main compares that 64-bit result against your strtoul value.bashfile system.outbashghidra system.out &bash# In Ghidra: rename variables, look at main() and make_secret()/hash()Expected output
system.out: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=63224e5a94fa31cb071c82105d8a70ffc806ac0b, for GNU/Linux 3.2.0, not stripped
What didn't work first
Tried: Run 'strings system.out' to find the expected hash value or password directly.
strings finds printable sequences in the file, and this hash is computed at runtime rather than stored. The 13 embedded bytes are XOR'd with 0xAA, so they sit in .rodata as non-printable bytes (c3 ff c8 c2 92 9b 8b c0 80 c2 c4 8b 00) and strings walks straight past them. Run or disassemble the binary.
Tried: Read the leaked bytes and stop there, assuming they are the number the server wants.
Over-reporting the length is a genuine leak (the heartbleed.c string in the binary is a wink at it): ask for 90 bytes and offsets 60..71 print as 105 85 98 104 56 49 33 106 42 104 110 33, the decoded string iUbh81!j*hn!. But that string is the hash input, not the hash. You still have to run it through djb2, or let the binary do it for you under GDB.
Learn more
In Ghidra, right-click on variable names like
local_padto rename them to something meaningful. You can also retype variables (e.g. changeundefined*tochar*) to make the decompiler output more readable.The program asks "how many bytes in length is your password" and then prints that many bytes of the heap buffer, so over-reporting the length reads past your input. Answer 90 and bytes 60 onward come back as 105 85 98 104 56 49 33 106 42 104 110 33 -86, the decoded secret string followed by the 0xAA-XOR'd terminator. That is the hash input; the comparison at the end is against djb2 of that string.
Step 2Recover the hash value by running system.out under GDB
ObservationThe hash function is deterministic and driven entirely by bytes baked into the binary. Break on it in GDB and read the 64-bit return value out of RAX rather than reimplementing the rolling hash.The simplest approach: run system.out under GDB, break on the hash() function, type anything for the prompts, then use 'finish' to return from hash(). The return value in RAX is the expected hash value, and it is a full 64-bit number: for this binary it is 15237662580160011234 (0xd3770d6251b31be2). Submit that number to the remote server.bashgdb ./system.outbash(gdb) break hashbash(gdb) runbash# When prompted, type anything for the passwordbash# For password length, type any numberbash# The breakpoint fires inside hash()bash(gdb) finishbash# After finish, RAX contains the return value (the expected hash)bash(gdb) print/u $raxbash# Note down the hash valuebash(gdb) quitbash# Now connect to the remote instance and submit that hash valuebashnc <HOST> <PORT_FROM_INSTANCE>What didn't work first
Tried: Break on main() instead of hash() and step through the entire program to find the hash.
Breaking on main and stepping through means hundreds of setup instructions before anything interesting. Break on the hash function by name instead, run, answer the prompts, and GDB stops exactly where you want it.
Tried: Read the hash from $eax (32-bit) instead of $rax (64-bit) after 'finish'.
hash() returns an unsigned long and main compares it with a 64-bit cmp against the strtoul result, so the top half is real data, not garbage. Truncating to $eax turns 15237662580160011234 into 1370692578, which the check rejects. Print $rax, and print it unsigned: the value has bit 63 set, so a signed print shows a negative number.
Learn more
The
finishcommand in GDB runs until the current function returns, then pauses. The hash is a 64-bit unsigned long, so the return value is in theRAXregister. Useprint/u $raxto read it as an unsigned number. This technique lets you use the program itself as a hash oracle - rather than reimplementing the hash algorithm, you let the binary compute it and just read the result.This works because the hash is deterministic: the obfuscation bytes embedded in the binary produce the same hash value every time. The exact value does not change between runs, so you can compute it locally with GDB and submit it to the remote server.
Step 3Submit the hash to the remote server
ObservationThe hash depends on those embedded bytes and not on your input, so the value you read locally is the one the server expects.Connect to the challenge server with netcat. Set any password, enter the password length (e.g. 1), then submit the hash value you obtained from GDB. The server will verify it and print the flag.bashnc <HOST> <PORT_FROM_INSTANCE>bash# Set a password: anythingbash# Enter password length: 1bash# Enter hash: <value from GDB RAX, e.g. 15237662580160011234>Learn more
The remote instance uses the same binary with the same embedded obfuscation bytes. Because the hash depends only on those embedded bytes (not on the password or anything else you provide), the hash value you computed locally is the same one the server expects.
Interactive tools
- Strings ExtractorPull printable text from any binary, library, or image. ASCII and UTF-16 detection, configurable minimum length, flag-like highlight, no command line needed.
- Hex ViewerView text or raw hex bytes as a xxd-style hex dump with byte offset, hex columns, and ASCII sidebar. Highlights printable characters and null bytes.
- File Magic IdentifierIdentify file types from magic numbers. Paste hex bytes or drop a file to detect PNG, JPEG, ZIP, PDF, ELF, PCAP, SQLite, and dozens of other formats.
Flag
Reveal flag
picoCTF{d0nt_trust_...}
The flag is the same for every instance. Run system.out under GDB, break on hash(), type anything for the prompts, then 'finish' and read RAX (a 64-bit unsigned long, 15237662580160011234 for this binary). Submit that number to the remote server to get the flag. Reading $eax instead of $rax truncates the value and the check rejects it. The flag is shown abbreviated on this page; work the steps above to recover the full value.