vault-door-7 picoCTF 2019 Solution

Description

1. Step 1Understand the bit manipulation

   Observation

   checkPassword in VaultDoor7.java combines characters into 32-bit integers with bit shifts and ORs. The password is packed, not transformed, so reversing the shifts recovers it completely.

   Open VaultDoor7.java. The checkPassword method packs four characters at a time into a 32-bit integer using bit shifts, then compares to hardcoded integers. You need to reverse this packing.

   bash

   cat VaultDoor7.java

   Copied!

   What didn't work first

   Tried: Look for a string literal or byte array in the source to read the password directly.

   There is no readable string constant. The hardcoded values are 32-bit integer literals that encode four characters each using bit shifts. You have to extract each byte from each integer mathematically - no string concatenation or array index trick will expose them.

   Tried: Assume the integers are ASCII codes and cast each one directly to a character.

   Each integer holds four packed characters, not one. Cast the whole 32-bit value to a char, or pass it to chr(), and you get a code point far outside printable ASCII. Right-shift by 24, 16, 8, and 0 and mask with 0xFF to pull out each byte.

   Learn more

   The encoding works by shifting each character's ASCII value to a specific bit position within a 32-bit integer. For example, four chars packed as: (c0 << 24) | (c1 << 16) | (c2 << 8) | c3.

   Bit shifting left by N is equivalent to multiplying by 2^N. OR-ing the shifted values combines them into a single integer without overlap, since each character occupies exactly 8 bits (one byte) within the 32-bit word.

2. Step 2Reverse the packing in Python

   Observation

   Each hardcoded integer holds exactly four characters, shifted left by 24, 16, 8, and 0 bits. A short Python script applying the inverse right shifts with a 0xFF mask pulls each byte back out.

   For each hardcoded 32-bit integer, extract each 8-bit group by right-shifting and masking with 0xFF. This recovers the four original characters per integer.

   python

   python3 -c "
   ints = [/* paste hardcoded integers */]
   password = ''
   for val in ints:
       password += chr((val >> 24) & 0xFF)
       password += chr((val >> 16) & 0xFF)
       password += chr((val >> 8) & 0xFF)
       password += chr(val & 0xFF)
   print(password)
   "

   Copied!

   What didn't work first

   Tried: Right-shift without masking, e.g. chr(val >> 24) for the first character.

   Without the 0xFF mask, sign bits or leftover high-order bits can bleed into the extracted value when the integer is negative or large. Masking after the shift keeps only the lowest 8 bits, giving a clean byte in 0-255 that chr() will accept.

   Tried: Extract bytes using little-endian order (shift 0, 8, 16, 24) instead of big-endian.

   Java packs big-endian: the leftmost character sits in the most significant byte, at shift 24. Reverse it little-endian and each group of four characters comes out backwards, so checkPassword fails. Match the order in the Java source: 24, 16, 8, 0.

   Learn more

   Masking with 0xFF (binary 11111111) after right-shifting isolates the lowest 8 bits of the result, discarding any higher bits. This is how you extract individual bytes from a multi-byte integer.

   This packing technique is common in low-level code for performance: processing 4 characters at once as a 32-bit word is faster than processing them individually on many architectures.

3. Step 3Submit the flag

   Observation

   Joining the extracted characters in order gives a readable ASCII string, which confirms the unpacking was right. All that remains is wrapping it in picoCTF{}.

   Concatenate all extracted characters in order to form the password. Wrap in picoCTF{...} to submit.

   Learn more

   When reversing bit manipulation, always check the shift amounts carefully in the original code. The order of bytes within the integer (big-endian vs little-endian) determines which byte to extract with which shift amount.

Flag

Reverse the 4-bytes-per-int packing scheme from VaultDoor7.java