The picoCTF Cryptography Roadmap: From Caesar to Elliptic Curves

The fastest way to learn crypto for CTF is in order

Cryptography is the one CTF category where guessing almost never works. You cannot fuzz your way into a flag, and reading a single writeup rarely transfers, because the next challenge uses a different scheme with a completely different attack. What does transfer is a map: knowing which family a challenge belongs to, what the standard attacks against that family are, and which one the author left the door open for.

This is that map. It orders every cryptography guide on this site into a single difficulty-tiered learning path, from reversing a Base64 string to recovering a private key from a reused elliptic-curve nonce. Each link below comes with a one-line summary of what it teaches and where it sits in the dependency graph. Read top to bottom the first time. After that, treat it as a lookup table: identify the scheme, jump to the tier, apply the attack.

In crypto you do not break the math. You find the one parameter the author chose badly and let the math do the rest.

If you are brand new, the only prerequisite is comfort reading a few lines of Python. The recipe-chain workflow covers the tooling. Everything else is built up tier by tier below.

The one decision that decides the whole challenge

Before you try a single attack, you have to identify the scheme. The attack you reach for is fully determined by which family the challenge belongs to. Misidentify the family and you will waste an hour applying RSA factoring tricks to what is actually a substitution cipher. Identify it correctly and the shortlist of viable attacks is usually two or three items long.

Run this triage in order. The first match wins:

If you see	It is probably	Go to tier
Only A-Z, 0-9, `=`, or hex-looking text	Encoding, not encryption	Tier 1
Letters only, word shapes preserved, no key math	Classical cipher	Tier 2
Fixed-length hex digest (32, 40, 64 chars)	Hash	Tier 3
Big integers named `n`, `e`, `c`	RSA	Tier 4
16-byte blocks, an IV, a server that decrypts	Symmetric (AES)	Tier 5
A shared generator `g`, a prime `p`, or a curve point	Key exchange (DH or ECC)	Tier 6
A seed, a token, or output from `rand()`	Weak randomness	Tier 7

Key insight: Encoding versus classical cipher versus symmetric versus RSA versus key exchange. That single classification decides everything that follows. Spend the first two minutes of any crypto challenge on identification, not on attacking. The shortlist of attacks falls out of the family for free.

The ordered path: seven tiers, easiest first

The tiers below are ordered by both difficulty and dependency. Later tiers assume the mental models built earlier (modular arithmetic from RSA underpins Diffie-Hellman, which underpins elliptic curves). Skip ahead if a tier is already familiar, but do not start at Tier 4 if Tier 1 is shaky. Each entry links the full guide and states, in one line, what it teaches.

Note: A realistic pace is one tier per week if you are also solving the practice challenges at the end of this post. The whole path is roughly a two-month on-ramp from zero to comfortable across every crypto subcategory picoCTF throws at you.

Tier 1: Encodings and triage

Encodings are not encryption. There is no key and nothing to break. The skill is recognition: spotting Base64, hex, ROT, URL encoding, or a chain of them, and reversing the stack in the right order. Most beginners lose time here by trying to decrypt something that only needs to be decoded.

Encodings in CTF teaches you to fingerprint Base64, hex, binary, ROT-N, and friends on sight, then peel a multi-layer encoding chain one step at a time.
The recipe-chain workflow teaches the tooling that turns triage into a repeatable pipeline, so a nested "magic" blob becomes a few clicks instead of a guessing game.

Tip: If text survives a round trip through a decoder unchanged, it was probably not that encoding. Let entropy guide you: printable ASCII with = padding is almost always Base64, while high-entropy bytes with no structure are more likely the output of a real cipher and belong in a later tier.

Tier 2: Classical ciphers

Classical ciphers are the historical schemes: Caesar, Vigenere, substitution, rail fence, and the keystream-flavored variants that lead toward modern stream ciphers. They fall to frequency analysis, known-plaintext, and a little patience. This tier teaches you to think about a cipher as a transformation you can invert by studying its statistics, which is the same instinct you will use against weak modern schemes.

Classical ciphers in CTF teaches frequency analysis, Kasiski examination, and the standard breaks for Caesar, Vigenere, and substitution ciphers.
Stream ciphers in CTF teaches why a reused keystream is fatal, how the many-time-pad attack works, and where XOR-based schemes leak.

Warning: A one-time pad is only unbreakable when the pad is truly one-time. The single most common CTF mistake (by design) is reusing a keystream across two messages, which turns an information-theoretically secure cipher into a solvable XOR puzzle. If you see two ciphertexts and one key, that is the intended break.

Tier 3: Hashes

Hashes are one-way functions, so you do not invert them. You either find a preimage by cracking (guessing inputs until a digest matches) or you abuse a structural weakness in how the hash was used. This tier sits between the symmetric and asymmetric worlds: the cracking half is about wordlists and rules, the structural half is about the Merkle-Damgard construction that lets you extend a signed message.

Hash cracking in CTF teaches identifying a digest by length and format, then driving hashcat and John the Ripper with the right wordlists and rules.
Hash length extension teaches the Merkle-Damgard weakness that lets you forge a valid MAC(secret || data || padding || extra) without ever knowing the secret.

Note: Length extension only bites naive hash(secret || message) constructions built on MD5, SHA-1, or SHA-2. HMAC and SHA-3 are immune. The moment a challenge authenticates with a raw concatenation MAC, length extension should be the first thing you reach for.

Tier 4: RSA

RSA is where the modular arithmetic gets real, and it is the single most common crypto family in CTF. The cryptosystem itself is sound. What gets broken is the parameter choice: a small exponent, a shared modulus, a tiny prime factor, a reusedn, or a message that was not padded. There are dozens of named RSA attacks, and the entire game is matching the symptoms in the challenge to the one attack the author set up.

RSA attacks in CTF teaches the full decision tree: factoring small or close primes, low-exponent and Hastad broadcast, Wiener for small d, common-modulus, and Franklin-Reiter related-message attacks.

Textbook RSA has no randomness. Every weakness in a CTF RSA challenge is the author handing you exactly one missing safeguard. Find which one, and the modulus does the rest.

Tip: Before any clever attack, paste n into a factoring database lookup and run a few seconds of small-prime trial division. A surprising number of CTF moduli are either already factored publicly or have a factor small enough to fall in seconds. Only when the easy checks fail do you move to the named attacks.

Tier 5: Symmetric encryption and AES

Modern symmetric crypto is AES, and AES the algorithm is not the target. The target is the mode of operation and how it is fed. ECB leaks structure, CBC leaks under a padding oracle, CTR and GCM collapse under nonce reuse. This tier teaches you to look past the cipher name and interrogate the mode, the IV, and whether the server tells you anything about why a decryption failed.

AES for CTF teaches the mode-of-operation map: the ECB cut-and-paste and byte-at-a-time attacks, CBC bit-flipping, and why nonce reuse breaks CTR and GCM.
The CBC padding oracle teaches the canonical interactive attack: how a single bit of leakage (valid padding or not) lets you decrypt an entire ciphertext without the key.

Warning: The padding oracle is the most important interactive attack in this tier. If a service decrypts your ciphertext and reacts differently to a padding error than to other errors, you can recover plaintext one byte at a time. Watch for distinct error messages, distinct timing, or distinct status codes.

Tier 6: Key exchange, Diffie-Hellman and elliptic curves

Key exchange is the top of the difficulty ladder, and it builds directly on the modular arithmetic from RSA. Diffie-Hellman lives in a multiplicative group modulo a prime; elliptic-curve cryptography moves the same idea onto a curve where the group operation is point addition. The attacks are about weak group parameters: smooth-order groups, small subgroups, invalid curves, and the nonce-reuse flaw that leaks an ECDSA private key outright.

Diffie-Hellman in CTF teaches the discrete-log problem, why a smooth prime order makes Pohlig-Hellman trivial, and how small-subgroup confinement leaks the shared secret.
Elliptic curves in CTF teaches the curve group law, the singular and invalid-curve attacks, anomalous-curve Smart attack, and the reused-nonce ECDSA key recovery.

Key insight: Diffie-Hellman and elliptic curves are the same idea in two different groups. Learn the discrete-log mindset once on DH and most of the ECC attacks become "the same flaw, new group law." Do not start ECC until DH feels comfortable.

Tier 7: Insecure randomness (cuts across every tier)

Randomness is not a tier so much as a flaw that hides inside all the others. An RSA key generated from a predictable seed, an AES nonce that repeats, an ECDSA signature reusingk, a session token from a known-state PRNG: every one of these is a cryptosystem undone by bad entropy rather than bad math. Save this guide for last, then keep it in mind everywhere.

Insecure randomness in CTF teaches how to recognize and predict weak PRNGs: seeding from the clock, recovering Mersenne Twister state from outputs, and the LCG recurrence recovery.

Note: A predictable seed collapses the keyspace from astronomically large to brute-forceable. If a challenge seeds from time() or a small integer, you are not attacking the cipher at all. You are enumerating a few thousand candidate seeds and checking which one reproduces the output.

Where to practice, easiest to hardest

Reading the guides builds the map. Solving challenges builds the instinct. These four picoCTF problems span the tiers in roughly increasing difficulty. Do them in order, and for each one, name the family from the triage table before you touch a tool.

picoCTF 2024 interencdec is pure Tier 1: a nested chain of encodings you peel one layer at a time. The whole challenge is recognition and ordering.
picoCTF 2024 rsa oracle is Tier 4: a decryption oracle you query to recover a message, the practical bridge between RSA theory and an interactive service.
picoCTF 2021 No Padding No Problem is Tier 4 with a twist: an unpadded RSA oracle that opens the door to a blinding or related-message attack.
picoCTF 2019 seed-spring is Tier 7: a predictable seed feeding a PRNG, where recovering the seed reproduces the entire output stream.

After these, the rest of the picoCTF crypto category falls into the same seven buckets. Identify the family, jump to the tier, apply the shortlist.

Quick reference

The seven-tier path in one screen

Encodings and triage. encodings, recipe chain. No key, just recognition.
Classical ciphers. classical ciphers, stream ciphers. Frequency analysis and keystream reuse.
Hashes. cracking, length extension. Preimages and Merkle-Damgard.
RSA. RSA attacks. Match the bad parameter to the named attack.
Symmetric. AES modes, padding oracle. Attack the mode, not the cipher.
Key exchange. Diffie-Hellman, elliptic curves. Discrete log in two groups.
Randomness. insecure randomness. Bad entropy undoes good math, everywhere.

Triage in one line

Printable with = is encoding. Letters-only is classical. Fixed-length hex is a hash. Big n, e, c is RSA. 16-byte blocks with an IV is AES. A generator or a curve point is key exchange. A seed or a token is weak randomness.

Learn the crypto families in this order and you stop staring at a blob wondering where to start. You glance, you classify, you open the right guide, and you attack the one parameter the author left exposed.