Hidden Cipher 2 picoCTF 2026 Solution

Description

1. Step 1Solve the math problem
   The program presents a math question. Solving it correctly produces a number, which is then used as the decryption key for the hidden cipher.
   bash

   ./binary

   Copied!

   bash

   cat problem.txt

   Copied!
   Learn more

   This challenge adds a key derivation step: the decryption key is not stored directly in the binary but is computed from the answer to a math problem. This pattern mirrors how real systems derive encryption keys from passwords using key derivation functions (KDFs) like PBKDF2, bcrypt, scrypt, and Argon2. The math problem here is trivially solvable by a human, unlike a real KDF which is deliberately computationally expensive.

   When a program presents a challenge (math, riddle, CAPTCHA) before revealing a secret, the first approach is to solve it correctly. If the problem is complex (e.g., a system of equations, a large prime factorisation), static analysis of the binary reveals either the expected answer hardcoded as a constant, or the algorithm used to check the answer - which you can then reverse or short-circuit. Dynamic analysis with ltrace or a breakpoint in GDB on the comparison instruction is often faster.

2. Step 2Use the math answer as the cipher key
   Read the source to see how the answer maps to a key. The relationship is read from the source, never assumed: source code is the only authoritative description of the key derivation.
   bash

   cat source.py

   Copied!

   bash

   cat source.c

   Copied!
   Learn more

   Key derivation is the process of transforming one value (a password, a shared secret, or as here, a math answer) into a cryptographic key. The transformation might be direct (the answer is the key), involve modular arithmetic (answer mod 256 for a single-byte XOR key), involve hashing (SHA-256 of the answer), or involve truncation or byte extraction.

   Walk through the full pipeline as the source describes it. As an illustration, suppose the math problem returns answer = 42. The script might:

   • key = answer (direct: 42 is used as the integer key, e.g. a Caesar shift of 42 mod 26 = 16)
   • key = answer % 256 (single-byte XOR key, here 42 = 0x2A)
   • key = sha256(str(answer).encode()).digest() (32-byte AES-256 key derived by hashing "42")

   The wrong derivation produces garbage even with the right answer. Read the entire pipeline in the source rather than guessing at the most common form. See the encodings guide for the broader pattern catalogue.

   Reading the source code is by far the fastest approach when it is available. Look for the section of code that follows the answer verification: it will apply some transformation to the answer variable and then use the result to call an encryption or decryption function. The transformation is the key derivation step. In Python source, this is immediately readable; in C source, look for bit operations, modulo operations, or hash function calls applied to the answer variable.

   In real malware and crimeware, the key derivation algorithm is often the most carefully hidden part of the binary. Analysts spend significant time reverse-engineering custom key schedules to be able to decrypt captured malware payloads or ransomware-encrypted files. Standard KDFs appear in legitimate software; non-standard ones are a red flag for malware.

3. Step 3Decrypt the ciphertext with the derived key
   Apply the cipher to the ciphertext using the key derived from the math answer.
   python

   python3 << 'EOF'
   math_answer = 42  # replace with actual answer
   # If the key is math_answer itself or derived from it:
   key = math_answer  # or bytes([math_answer % 256]) etc.
   ct  = bytes.fromhex("CIPHERTEXT_HEX")
   pt  = bytes(c ^ key for c in ct)
   print(pt.decode())
   EOF

   Copied!
   Learn more

   Once you know the key derivation formula and the ciphertext, decryption is mechanical. The example script XORs every byte of the ciphertext with a single-byte key derived from the math answer. If the key is multi-byte, cycle it with zip(ct, itertools.cycle(key_bytes)). If the cipher is a Caesar shift, apply (c - shift) % 26 to each alphabetic character.

   A useful trick for single-byte XOR: if you know part of the plaintext (like picoCTF{), you can recover the key byte directly by XORing the known plaintext with the corresponding ciphertext byte - no brute-forcing needed. This is the known-plaintext attack, and it works because XOR is reversible: C XOR P = K just as C XOR K = P.

   This challenge demonstrates why security through complexity (hiding the key derivation algorithm) is not security. An attacker with source code or a disassembler recovers the algorithm in minutes. Real security requires algorithms that are secure even when the adversary knows every detail of how they work - this is Kerckhoffs's principle, a foundational tenet of cryptography.

Alternate Solution

Flag

The math problem's answer serves as the decryption key - solve the math, use the answer to decrypt.