Description
We found a brand new type of encryption. Can you break it? The ciphertext is: kjlijdliljhdjdhfkfkhhjkkhhkihlhnhghekfhmhjhkhfhekfkkkjkghghjhlhghmhhhfkikfkfhm
Setup
Download new_caesar.py from the challenge page to understand the encoding scheme.
Solution
- Step 1Understand the encodingRead new_caesar.py. Each character is converted to a value 0-15 (its nibbles), and each nibble is encoded as a letter a-p. A key-based Caesar shift is then applied to each encoded character. The full alphabet used is a-p (16 characters).
Learn more
The "new Caesar" scheme is a two-stage encoding. In the first stage, each byte of the plaintext is split into two nibbles (4-bit halves) -- the high nibble (upper 4 bits, value 0–15) and the low nibble (lower 4 bits, value 0–15). Each nibble is then mapped to a letter from the alphabet
a–p(16 letters), so one input character becomes two output characters.The second stage applies a Caesar shift to each encoded character using a single-character key. The shift is the key letter's position in the
a–palphabet. This means the combined cipher is not just a Caesar cipher -- it's a Caesar cipher on top of a base-16 encoding, applied at the nibble level rather than the character level.Why this matters: The encoding expands the ciphertext (each byte becomes two characters) and restricts the alphabet to just 16 letters, which shrinks the key space. Instead of 26 possible Caesar shifts, there are only 16. The design demonstrates how layering encoding on top of a cipher does not add significant security if the layers are independently weak.
- Step 2Brute-force the keyThere are only 16 possible key values (letters a through p). Try each one, reverse the Caesar shift, reverse the nibble encoding, and check whether the result is valid printable ASCII.python3 << 'EOF' ALPHABET = "abcdefghijklmnop" def b16_decode(encoded): result = [] for i in range(0, len(encoded), 2): hi = ALPHABET.index(encoded[i]) lo = ALPHABET.index(encoded[i+1]) result.append(chr((hi << 4) | lo)) return "".join(result) def shift_char(c, key, decrypt=True): idx = ALPHABET.index(c) key_idx = ALPHABET.index(key) if decrypt: return ALPHABET[(idx - key_idx) % len(ALPHABET)] return ALPHABET[(idx + key_idx) % len(ALPHABET)] ciphertext = "kjlijdliljhdjdhfkfkhhjkkhhkihlhnhghekfhmhjhkhfhekfkkkjkghghjhlhghmhhhfkikfkfhm" for key in ALPHABET: try: unshifted = "".join(shift_char(c, key) for c in ciphertext) decoded = b16_decode(unshifted) if all(32 <= ord(c) < 127 for c in decoded): print(f"Key={key}: {decoded}") except Exception: pass EOF
Learn more
Brute-force is viable here because the key space is tiny -- only 16 possibilities. For each candidate key, you reverse the Caesar shift on the ciphertext, then attempt to decode the result as base-16 nibble pairs. If the decoded bytes all fall in the printable ASCII range (32–126), the key is almost certainly correct.
The printable ASCII check acts as a crib -- a known property of the plaintext. The flag must consist of printable characters (letters, digits, braces, underscores), so any key that produces garbage bytes is immediately eliminated. This kind of filter is standard practice when brute-forcing substitution ciphers: you check candidate plaintexts against an expected character set rather than manually examining all 16 outputs.
Bit manipulation: The decode step uses
(hi << 4) | lo-- shifting the high nibble left by 4 bits and OR-ing with the low nibble to reassemble the original byte. This is the inverse of splitting a byte into two nibbles usingbyte >> 4andbyte & 0x0F. Bit manipulation at the nibble level is fundamental to understanding how many encoding schemes and simple ciphers operate on binary data.
Flag
picoCTF{...}
The custom encoding is a base-16 representation followed by a per-character shift -- only 16 possible keys to try.