vault-door-4 picoCTF 2019 Solution

Description

1. Step 1

   Read the Java source and identify the four encodings

   Observation

   I noticed the challenge description mentioned bytes written in different number bases, which suggested I needed to inspect the Java source directly to identify each literal form (decimal, hex, octal, and char) before attempting any decoding.

   Open VaultDoor4.java and find the checkPassword method. It builds a byte array called myBytes where each slot holds an ASCII code written in one of four Java literal forms: plain decimal (e.g. 106), hex with 0x prefix (e.g. 0x55), octal with a leading 0 (e.g. 0142), or a raw char literal (e.g. '7'). The method then compares each character of your input against the decoded value.

   bash

   cat VaultDoor4.java

   Copied!

   What didn't work first

   Tried: Run strings VaultDoor4.java hoping to spot the password directly in the output

   strings will print the human-readable text of the source file, but the password characters are encoded as numeric literals spread across four different bases, so no contiguous ASCII password string appears. The correct approach is to parse and convert each integer literal individually rather than treating the file as raw binary.

   Tried: Assume all entries in myBytes are plain decimal and convert them directly with chr()

   Entries with a 0x prefix are hexadecimal and entries with a leading 0 are octal, so treating them as decimal yields wrong character values - for example 0x55 read as decimal 55 gives '7' instead of 'U'. You must detect the prefix of each literal before calling chr() on it.

   Learn more

   In Java (and C), an integer literal that starts with 0x is hexadecimal, a literal that starts with a lone 0 followed by more digits is octal, and a literal with no special prefix is decimal. A char literal like '7' is already the character itself and needs no conversion at all.

   The four groups in myBytes are roughly: the first 8 entries are decimal, the next 8 are hex, the next 8 are octal, and the last 8 are char literals. Each group still encodes a printable ASCII character - the difference is only in how the author chose to write the number down in source code.

   This multi-base trick defeats a naive strings search while remaining trivially reversible once you recognize the notation.

2. Step 2

   Decode all four groups with Python

   Observation

   I noticed that Python shares the 0x hex prefix with Java and supports octal via the 0o prefix, which suggested it was the most natural scripting tool to convert all four literal forms to ASCII characters with a single chr() call per value.

   Python understands all four literal forms natively. Copy the values from the source file, paste them into a Python list exactly as written (Python uses the same 0x and 0 prefix conventions as Java, and handles single-quoted chars as strings), then call chr() on each integer to recover the password characters.

   python

   python3 -c "
   # Paste the four groups directly from VaultDoor4.java.
   # Decimal group
   dec = [106, 85, 53, 116, 95, 52, 95, 98]
   # Hex group (0x prefix - Python reads these natively)
   hex_ = [0x55, 0x6e, 0x43, 0x68, 0x5f, 0x30, 0x66, 0x5f]
   # Octal group (0o prefix in Python; Java uses bare 0 prefix)
   oct_ = [0o142, 0o131, 0o164, 0o63, 0o163, 0o137, 0o66, 0o61]
   # Char-literal group (already the characters themselves)
   chars = list('e0f27690')  # replace with the chars from your file
   print(''.join(chr(v) for v in dec + hex_ + oct_) + ''.join(chars))
   "

   Copied!

   Expected output

   picoCTF{...}

   What didn't work first

   Tried: Paste Java octal literals with their bare leading-zero prefix directly into Python 3 (e.g. 0142 instead of 0o142)

   Python 3 raises a SyntaxError on integer literals with a bare leading zero because that notation was removed after Python 2. You must manually replace each leading-zero octal like 0142 with the explicit 0o prefix form 0o142 before the interpreter will accept it.

   Tried: Concatenate the four groups in the wrong order - for example alphabetical or hex-before-decimal

   The myBytes array is indexed sequentially: positions 0-7 are decimal, 8-15 are hex, 16-23 are octal, and 24-31 are char literals. Reordering the groups produces a string that passes none of the character comparisons in checkPassword, because the vault checks each index individually against the corresponding decoded byte.

   Learn more

   Python's chr() converts any integer (decimal, hex, or octal) to its Unicode/ASCII character. Because Python shares the 0x hex prefix with Java you can paste hex literals verbatim. For octal you need to swap Java's leading-zero notation (e.g. 0142) for Python's explicit0o prefix (e.g. 0o142), because Python 3 treats a bare leading zero as a syntax error.

   The char-literal group is already decoded: just read the characters directly from the source code and concatenate them.

3. Step 3

   Submit the password

   Observation

   I noticed the checkPassword method compares each myBytes index in sequential order, which confirmed that the decoded characters must be concatenated in array order (decimal, then hex, then octal, then char literals) to form the correct 32-character password.

   Concatenate the decoded characters in array order - decimal group first, then hex, then octal, then char literals - to form the 32-character password string. Wrap it in picoCTF{...} to submit the flag.

   Learn more

   The flag format for vault-door challenges is picoCTF{password}. Preserve the exact order from the myBytes array: the first 8 decoded characters come from the decimal group, the next 8 from hex, the next 8 from octal, and the final 8 from the char literals.

Flag

The myBytes array mixes four literal forms: decimal, 0x hex, leading-zero octal, and char literals. Convert each to its ASCII character (Python chr() handles decimal, hex, and octal; char literals are already decoded) and concatenate in order.