Hidden Cipher 2 picoCTF 2026 Solution

Description

1. Step 1Connect via netcat and solve the math problem

   Observation

   The server poses a math problem before it prints anything. Connect and answer it correctly, and the encoded flag array follows.

   Connect to the challenge server. It presents a math question like 'What is 3 - 2?'. Answer it correctly. The server then prints the encoded flag as an array of numbers.

   bash

   nc <HOST> <PORT_FROM_INSTANCE>

   Copied!

   What didn't work first

   Tried: Trying to skip the math problem by sending a blank line or random input

   The server checks your answer before printing anything: a blank line or a wrong number gets an 'Incorrect' and a closed connection. The math is a gate, not decoration. Answer it and the array follows.

   Tried: Running the downloaded binary locally instead of connecting to the remote server

   The local binary accepts the same input, but the flag it encodes is a placeholder. The real one exists only on the competition server, so take the encoded output from there.

   Learn more

   This challenge adds a key derivation step: the decryption key is not stored directly in the binary but is computed from the answer to a math problem. This pattern mirrors how real systems derive encryption keys from passwords using key derivation functions (KDFs) like PBKDF2, bcrypt, scrypt, and Argon2. The math problem here is trivially solvable by a human, unlike a real KDF which is deliberately computationally expensive.

   When a program presents a challenge (math, riddle, CAPTCHA) before revealing a secret, the first approach is to solve it correctly. If the problem is complex (e.g., a system of equations, a large prime factorisation), static analysis of the binary reveals either the expected answer hardcoded as a constant, or the algorithm used to check the answer - which you can then reverse or short-circuit. Dynamic analysis with ltrace or a breakpoint in GDB on the comparison instruction is often faster.

2. Step 2Understand the encoding: each flag character is multiplied by the math answer

   Observation

   The array holds integers far larger than any ASCII value, so the characters were scaled by some factor. Open the binary in Ghidra and read encode_flag() to confirm the multiplication.

   Load the binary in Ghidra and look at the encode_flag() function. It takes the flag string and the math answer, then outputs each character's ASCII value multiplied by the answer. To decrypt: divide each number in the output by the answer.

   bash

   ghidraRun &   # then create a project and import hiddencipher2

   Copied!

   bash

   # In Ghidra: find encode_flag() to confirm the multiplication

   Copied!

   What didn't work first

   Tried: Using 'strings hiddencipher2' to find the key or the algorithm without opening Ghidra

   strings shows format strings and error messages, and the multiplication constant is an integer embedded in machine code rather than a printable string, so it never appears. Disassemble encode_flag() and read the multiply instruction.

   Tried: Assuming the key derivation is XOR or Caesar shift after seeing the multiplication in Ghidra and reaching for an XOR solver

   This multiplies each ASCII value by the answer. It is not XOR and not a Caesar shift, so an XOR brute-forcer or a constant subtraction produces garbage. The inverse is integer division.

   Learn more

   Key derivation is the process of transforming one value (a password, a shared secret, or as here, a math answer) into a cryptographic key. The transformation might be direct (the answer is the key), involve modular arithmetic (answer mod 256 for a single-byte XOR key), involve hashing (SHA-256 of the answer), or involve truncation or byte extraction.

   Walk through the full pipeline as the source describes it. As an illustration, suppose the math problem returns answer = 42. The script might:

   • key = answer (direct: 42 is used as the integer key, e.g. a Caesar shift of 42 mod 26 = 16)
   • key = answer % 256 (single-byte XOR key, here 42 = 0x2A)
   • key = sha256(str(answer).encode()).digest() (32-byte AES-256 key derived by hashing "42")

   The wrong derivation produces garbage even with the right answer. Read the entire pipeline in the source rather than guessing at the most common form. See the encodings guide for the broader pattern catalogue.

   Reading the source code is by far the fastest approach when it is available. Look for the section of code that follows the answer verification: it will apply some transformation to the answer variable and then use the result to call an encryption or decryption function. The transformation is the key derivation step. In Python source, this is immediately readable; in C source, look for bit operations, modulo operations, or hash function calls applied to the answer variable.

   In real malware and crimeware, the key derivation algorithm is often the most carefully hidden part of the binary. Analysts spend significant time reverse-engineering custom key schedules to be able to decrypt captured malware payloads or ransomware-encrypted files. Standard KDFs appear in legitimate software; non-standard ones are a red flag for malware.

3. Step 3Decode the flag by dividing each output number by the math answer

   Observation

   encode_flag() multiplies each character's ASCII value by the math answer, so the inverse is integer division. One Python line recovers every character.

   Take the array of numbers from the server output and divide each by the math answer to get the original ASCII values. Convert to characters to read the flag.

   python

   python3 << 'EOF'
   math_answer = 3   # replace with the answer to the math question you got
   
   # Copy the array of numbers from the server output here:
   encoded = [336, 315, 297, ...]   # replace with actual numbers
   
   flag = ''.join(chr(n // math_answer) for n in encoded)
   print(flag)
   EOF

   Copied!

   bash

   # If the math answer happens to be 1, the numbers ARE the ASCII values directly:

   Copied!

   bash

   # python3 -c "print(''.join(chr(n) for n in [112, 105, 99]))"

   Copied!

   Expected output

   picoCTF{m4th_b3h1nd_c1ph3r_...}

   What didn't work first

   Tried: Using modulo (n % math_answer) instead of integer division (n // math_answer) to recover ASCII values

   Modulo returns the remainder, not the quotient: 336 mod 3 is 0, where the quotient 112 is the 'p' you want. Every position decodes to zero or some tiny number, giving null bytes and control characters. Use integer division.

   Tried: Hardcoding math_answer = 1 in the decode script without re-solving the math problem each session

   The math problem changes between connections, so the multiplier changes with it. Hardcode the wrong answer and every division lands off, producing a garbled flag. Read the answer from the session you are actually in.

   Learn more

   Once you know the encoding and the ciphertext, decryption is mechanical. Here the encoder simply multiplies each flag byte's ASCII value by the math answer, so the inverse is integer division: chr(n // answer) for each number n in the output array. This is not an XOR or a shift cipher - the operation is plain multiplication, so do not reach for an XOR tool or modular subtraction.

   A useful sanity check: if the math answer came out as 1, multiplication is a no-op and the output numbers are the raw ASCII codes already (a session that happens to draw 3 - 2 = 1 can read the flag straight off). For any other answer, divide every number by it. If a division leaves a remainder, your answer value is wrong - re-read it from the prompt.

   This challenge demonstrates why security through complexity (hiding the key derivation algorithm) is not security. An attacker with source code or a disassembler recovers the algorithm in minutes. Real security requires algorithms that are secure even when the adversary knows every detail of how they work - this is Kerckhoffs's principle, a foundational tenet of cryptography.

Flag

Connect via netcat, solve the math problem (e.g. '3 - 2 = 1'). The server outputs each flag character multiplied by your answer. Divide each number by the answer to recover the original ASCII characters.