Skip to main content
picoCTF Solutions
picoCTF 2025Reverse EngineeringMedium

Binary Instrumentation 2 picoCTF 2025 Solution

Frida hooks correct the path the binary writes to and capture the WriteFile buffer, which turns out to hold the flag as Base64.

Published: April 2, 2025Updated: August 25, 2026

Description

A Windows binary claims to write the flag to flag.txt, but the file path is wrong and WriteFile writes zero bytes because nNumberOfBytesToWrite is 0. Use Frida to hook CreateFile and WriteFile, fix the path, and intercept the flag data before it is written.

Setup

Download bininst2.zip

Unzip with the password picoctf and confirm the binary with file bininst2.exe.

Run bininst2.exe to see it runs but produces no visible output and no flag.txt.

Install Frida (pip install frida-tools) on a Windows machine.

Use frida-trace -i CreateFile -i WriteFile bininst2.exe to auto-generate handler stubs, then edit those handlers to intercept and fix the calls.

bash
pip install frida-tools
bash
# Auto-generate handler stubs:
bash
frida-trace -i CreateFile -i WriteFile bininst2.exe
bash
# This creates a __handlers__ folder with JS files for each function

Solution

Want to try it yourself first?

The guided walkthrough reveals hints one step at a time.

Walk me through it
  1. Step 1Hook CreateFile to fix the path
    Observation
    Running the binary produces no flag.txt and no output, so the file creation is failing. Tracing CreateFileW shows a literal placeholder path, so intercept and replace that argument before the call goes through.
    The auto-generated CreateFile handler shows the binary is passing <insert path here> as the filename - a literal placeholder that will always fail. Edit the CreateFileW handler to print the filename argument and replace it with flag.txt so the file can be created.
    js
    // In __handlers__/kernel32.dll/CreateFileW.js
onEnter(log, args, state) {
  log("CreateFileW called, filename: " + args[0].readUtf16String());
  // Replace the broken path with flag.txt
  var newPath = Memory.allocUtf16String("flag.txt");
  this.newPath = newPath;  // keep a reference so it stays alive
  args[0] = newPath;
},
    What didn't work first

    Tried: Reading args[0] with readUtf8String() instead of readUtf16String()

    Windows API W-suffix functions (CreateFileW) pass strings as UTF-16 wide characters. readUtf8String() treats each byte independently, so most characters appear garbled or empty and the path printed is meaningless. readUtf16String() reads two bytes per character, matching the actual encoding Windows uses.

    Tried: Skipping the this.newPath assignment and only setting args[0] = newPath inside onEnter

    Memory.allocUtf16String() returns a pointer into a Frida-managed allocation. With no lasting reference on the JavaScript side, the collector can free it before CreateFileW reads it, producing a crash or a corrupted path. Keep the pointer on this so the allocation survives the call.

    Learn more

    frida-trace is a command-line tool that auto-generates Frida handler stubs for named functions. Running frida-trace -i CreateFile -i WriteFile bininst2.exe spawns the process and creates JavaScript files in a __handlers__ folder - one per intercepted function. The stubs just log calls initially; you edit them to add custom logic like reading or replacing arguments.

    The args[0].readUtf16String() call reads the first argument to CreateFileW as a wide (UTF-16) string, which is how Windows API functions expect string parameters. Setting args[0] = newPath replaces the pointer with a new allocation, redirecting the file creation to the correct path. The this.newPath assignment keeps a JavaScript reference alive so the native memory is not garbage-collected before the function finishes.

  2. Step 2Hook WriteFile to capture the flag data
    Observation
    With the path fixed, flag.txt is still empty, so the write itself is broken: the byte count passed to WriteFile is zero. Read the buffer from the hook instead of waiting for the write.
    The WriteFile call has a bug where nNumberOfBytesToWrite is 0, so no bytes are written at all even though the buffer contains the full flag. Hook WriteFile to print the buffer before it is written. The buffer contains the Base64-encoded flag.
    js
    // In __handlers__/kernel32.dll/WriteFile.js
onEnter(log, args, state) {
  var buf = args[1];
  // args[2] (nNumberOfBytesToWrite) is 0 because of the bug, so reading
  // that many bytes returns an empty string. Read to the null terminator.
  log("WriteFile buffer: " + buf.readUtf8String());
},
    bash
    # Re-run frida-trace with the edited handlers:
    bash
    frida-trace -i CreateFile -i WriteFile bininst2.exe
    What didn't work first

    Tried: Reading the WriteFile buffer with readUtf16String() instead of readUtf8String()

    WriteFile is the non-Unicode variant, and its buffer holds raw bytes, here an ASCII Base64 string, not wide characters. readUtf16String() pairs those bytes into UTF-16 code points, so the output is garbled and short. readUtf8String reads exactly the bytes as they are.

    Tried: Printing args[2].toInt32() to verify the byte count before reading the buffer

    The bug is that the byte count argument is zero at call time, which is why nothing reaches disk. Reading it confirms the zero, but passing that zero as the read length gives you an empty string. Call readUtf8String() with no length so it reads to the null terminator, or hardcode a generous length, rather than trusting the broken argument.

    Learn more

    The WriteFile Windows API takes a handle, a buffer pointer, a byte count, and an output pointer for bytes written. When the byte count is zero, the file is created but completely empty. Hooking the function and printing the buffer contents at call time shows the full data the binary intended to write, even though the write itself fails to store it correctly.

    This technique of intercepting file writes to capture data before it lands on disk is a standard malware analysis technique. Encrypted ransomware, for example, passes plaintext data through WriteFile before encrypting it to disk; hooking WriteFile lets an analyst recover the original files.

  3. Step 3Decode the Base64 flag
    Observation
    The buffer is a long alphanumeric string ending in = padding, with a length divisible by four. That is Base64; decode it.
    The WriteFile buffer contains a Base64 string. Decode it to recover the picoCTF flag.
    bash
    # The buffer output will be something like:
    bash
    # cGljb0NURntmcjFkYV9mMHJfYjFuX2luNXRydW0zbnQ0dGlvbiFfYjIxYS4uLn0=
    bash
    echo 'cGljb0NURntmcjFkYV9mMHJfYjFuX2luNXRydW0zbnQ0dGlvbiFfYjIxYS4uLn0=' | base64 -d

    Expected output

    picoCTF{fr1da_f0r_b1n_in5trum3nt4tion!_b21a...}
    Learn more

    The binary encodes the flag as Base64 before writing it, which is why the file would be unreadable even if WriteFile worked correctly. By intercepting the buffer in the WriteFile hook, you capture the encoded bytes and can decode them directly without needing to fix the write itself.

Interactive tools
  • Strings ExtractorPull printable text from any binary, library, or image. ASCII and UTF-16 detection, configurable minimum length, flag-like highlight, no command line needed.
  • Hex ViewerView text or raw hex bytes as a xxd-style hex dump with byte offset, hex columns, and ASCII sidebar. Highlights printable characters and null bytes.

Flag

Reveal flag

picoCTF{fr1da_f0r_b1n_in5trum3nt4tion!_b21a...}

Use frida-trace to auto-generate handler stubs, edit CreateFile to fix the filename and WriteFile to print the buffer, then decode the Base64 output.

Key takeaway

Hooking at the system call boundary shows every interaction a process has with the operating system: file I/O, network calls, memory allocation. Data has to pass through those APIs in raw form before it is transformed or stored, so hooking them exposes what encryption, encoding, or a deliberate bug would otherwise hide. That is why WriteFile and CreateFile hooks are foundational in malware analysis, from ransomware forensics to DRM research.

Related reading

Useful tools for Reverse Engineering

Where to go next

Try these next