Hidden Cipher 2 picoCTF 2026 Solution

Description

1. Step 1

   Connect via netcat and solve the math problem

   Observation

   I noticed the setup described a remote server that presents a math problem before revealing any output, which suggested the first action must be connecting via netcat and correctly answering the gate question to receive the encoded flag array.

   Connect to the challenge server. It presents a math question like 'What is 3 - 2?'. Answer it correctly. The server then prints the encoded flag as an array of numbers.

   bash

   nc <HOST> <PORT_FROM_INSTANCE>

   Copied!

   What didn't work first

   Tried: Trying to skip the math problem by sending a blank line or random input

   The server checks the answer before printing anything useful - sending a blank line or a wrong number causes it to print 'Incorrect' and close the connection. The math problem is a gate, not decoration. Solve it correctly (e.g. '3 - 2 = 1') and the encoded array appears immediately after.

   Tried: Running the downloaded binary locally instead of connecting to the remote server

   The local binary may accept the same math input, but the flag it encodes is a placeholder or a local test value, not the real picoCTF flag. The real flag only exists on the competition server. Always use the nc command with the provided host and port to get the real encoded output.

   Learn more

   This challenge adds a key derivation step: the decryption key is not stored directly in the binary but is computed from the answer to a math problem. This pattern mirrors how real systems derive encryption keys from passwords using key derivation functions (KDFs) like PBKDF2, bcrypt, scrypt, and Argon2. The math problem here is trivially solvable by a human, unlike a real KDF which is deliberately computationally expensive.

   When a program presents a challenge (math, riddle, CAPTCHA) before revealing a secret, the first approach is to solve it correctly. If the problem is complex (e.g., a system of equations, a large prime factorisation), static analysis of the binary reveals either the expected answer hardcoded as a constant, or the algorithm used to check the answer - which you can then reverse or short-circuit. Dynamic analysis with ltrace or a breakpoint in GDB on the comparison instruction is often faster.

2. Step 2

   Understand the encoding: each flag character is multiplied by the math answer

   Observation

   I noticed the server output was an array of integers much larger than typical ASCII values, which suggested the characters had been scaled by a numeric factor and pointed toward opening the binary in Ghidra to find the encode_flag() function and confirm the multiplication algorithm.

   Load the binary in Ghidra and look at the encode_flag() function. It takes the flag string and the math answer, then outputs each character's ASCII value multiplied by the answer. To decrypt: divide each number in the output by the answer.

   bash

   ghidra hiddencipher2 &

   Copied!

   bash

   # In Ghidra: find encode_flag() to confirm the multiplication

   Copied!

   What didn't work first

   Tried: Using 'strings hiddencipher2' to find the key or the algorithm without opening Ghidra

   The strings output shows string literals like format strings and error messages, but the multiplication constant is an integer embedded in the machine code, not a printable string. It will not appear in 'strings' output. You need Ghidra (or objdump/radare2) to disassemble the encode_flag() function and read the IMUL or MUL instruction that applies the key.

   Tried: Assuming the key derivation is XOR or Caesar shift after seeing the multiplication in Ghidra and reaching for an XOR solver

   This cipher multiplies each character's ASCII value by the math answer - it is not XOR and not a Caesar shift. Plugging the output numbers into an XOR brute-force tool or subtracting a constant will produce garbage. The inverse operation is integer division: chr(n // answer) for each number n in the output array.

   Learn more

   Key derivation is the process of transforming one value (a password, a shared secret, or as here, a math answer) into a cryptographic key. The transformation might be direct (the answer is the key), involve modular arithmetic (answer mod 256 for a single-byte XOR key), involve hashing (SHA-256 of the answer), or involve truncation or byte extraction.

   Walk through the full pipeline as the source describes it. As an illustration, suppose the math problem returns answer = 42. The script might:

   • key = answer (direct: 42 is used as the integer key, e.g. a Caesar shift of 42 mod 26 = 16)
   • key = answer % 256 (single-byte XOR key, here 42 = 0x2A)
   • key = sha256(str(answer).encode()).digest() (32-byte AES-256 key derived by hashing "42")

   The wrong derivation produces garbage even with the right answer. Read the entire pipeline in the source rather than guessing at the most common form. See the encodings guide for the broader pattern catalogue.

   Reading the source code is by far the fastest approach when it is available. Look for the section of code that follows the answer verification: it will apply some transformation to the answer variable and then use the result to call an encryption or decryption function. The transformation is the key derivation step. In Python source, this is immediately readable; in C source, look for bit operations, modulo operations, or hash function calls applied to the answer variable.

   In real malware and crimeware, the key derivation algorithm is often the most carefully hidden part of the binary. Analysts spend significant time reverse-engineering custom key schedules to be able to decrypt captured malware payloads or ransomware-encrypted files. Standard KDFs appear in legitimate software; non-standard ones are a red flag for malware.

3. Step 3

   Decode the flag by dividing each output number by the math answer

   Observation

   I noticed that Ghidra confirmed encode_flag() performs a straight multiplication of each character's ASCII value by the math answer, which meant the inverse operation is integer division and a short Python one-liner using chr(n // math_answer) would recover every original character.

   Take the array of numbers from the server output and divide each by the math answer to get the original ASCII values. Convert to characters to read the flag.

   python

   python3 << 'EOF'
   math_answer = 3   # replace with the answer to the math question you got
   
   # Copy the array of numbers from the server output here:
   encoded = [339, 294, 294, ...]   # replace with actual numbers
   
   flag = ''.join(chr(n // math_answer) for n in encoded)
   print(flag)
   EOF

   Copied!

   bash

   # If the math answer happens to be 1, the numbers ARE the ASCII values directly:

   Copied!

   bash

   # python3 -c "print(''.join(chr(n) for n in [339, 294, ...])"

   Copied!

   Expected output

   picoCTF{m4th_b3h1nd_c1ph3r_...}

   What didn't work first

   Tried: Using modulo (n % math_answer) instead of integer division (n // math_answer) to recover ASCII values

   Modulo gives the remainder of the division, not the quotient - for example 339 % 3 = 0, not 113 ('q'). Every character decodes to 0 or a tiny number, producing null bytes or control characters instead of readable text. The correct inverse of multiplication is integer division with //.

   Tried: Hardcoding math_answer = 1 in the decode script without re-solving the math problem each session

   The math problem may vary between connections - the server can ask '3 - 2' (answer 1) or '5 * 3' (answer 15) or other combinations. If your hardcoded answer is wrong, every division produces a non-integer result that chr() rounds to the wrong character, giving a garbled flag. Always read the answer from the current session's server output before running the decode script.

   Learn more

   Once you know the encoding and the ciphertext, decryption is mechanical. Here the encoder simply multiplies each flag byte's ASCII value by the math answer, so the inverse is integer division: chr(n // answer) for each number n in the output array. This is not an XOR or a shift cipher - the operation is plain multiplication, so do not reach for an XOR tool or modular subtraction.

   A useful sanity check: if the math answer came out as 1, multiplication is a no-op and the output numbers are the raw ASCII codes already (the video happened to draw 3 - 2 = 1 and could read the flag straight off). For any other answer, divide every number by it. If a division leaves a remainder, your answer value is wrong - re-read it from the prompt.

   This challenge demonstrates why security through complexity (hiding the key derivation algorithm) is not security. An attacker with source code or a disassembler recovers the algorithm in minutes. Real security requires algorithms that are secure even when the adversary knows every detail of how they work - this is Kerckhoffs's principle, a foundational tenet of cryptography.

Flag

Connect via netcat, solve the math problem (e.g. '3 - 2 = 1'). The server outputs each flag character multiplied by your answer. Divide each number by the answer to recover the original ASCII characters.