asm1 picoCTF 2019 Solution

Description

1. Step 1

   Read the assembly and set up the stack frame

   Observation

   I noticed the challenge provides a raw .S assembly file and asks for a specific return value, which suggested the first step is reading the source to understand the function structure and identify where the argument (0x345) lives in the x86 cdecl calling convention.

   Open test.S. The function asm1 takes one argument (0x345 = 837). Trace through it manually: set up the stack frame in your head (or on paper), tracking the value of eax and other registers at each instruction.

   bash

   cat test.S

   Copied!

   What didn't work first

   Tried: Running 'objdump -d test.S' to disassemble and read the function flow.

   objdump -d disassembles compiled binaries (ELF/PE), not raw .S source files. Running it on an uncompiled .S file produces no output or an error. The correct approach is to simply read the .S file with cat or a text editor, since it is already human-readable AT&T syntax assembly.

   Tried: Assuming the first argument is in eax or edi (System V AMD64 convention) rather than on the stack.

   This is 32-bit x86 cdecl, not the 64-bit System V ABI. In cdecl, arguments are pushed onto the stack before the call, so the first argument lives at [ebp+8] after 'push ebp; mov ebp, esp'. Looking in edi or eax at function entry gives a garbage value and produces a wrong trace.

   Learn more

   In x86 calling convention (cdecl), arguments are pushed on the stack right-to-left. The first argument is at [ebp+8] after the function prologue (push ebp; mov ebp, esp). The return value is in eax when the function returns.

   Key x86 instructions: cmp a, b sets flags based on a - b. jg jumps if greater (signed). jl jumps if less. je jumps if equal. add and sub modify a register. mov copies a value.

2. Step 2

   Trace the function logic

   Observation

   I noticed the function uses cmp instructions against hardcoded values followed by conditional jumps (jg, jl, je), which suggested manually following the branch taken for the input 0x345 would determine what eax holds before ret.

   Start with the argument value 0x345 in [ebp+8]. Follow each branch condition (compare the argument to hardcoded values) to determine which branch is taken. Track the final value loaded into eax before ret.

   Learn more

   Compile the assembly and run it to verify your answer: gcc -m32 -o test test.S -no-pie && python3 -c "import ctypes; lib=ctypes.CDLL('./test'); print(hex(lib.asm1(0x345)))". This is often faster than manual tracing for complex functions.

3. Step 3

   Submit the return value as the flag

   Observation

   I noticed the problem statement says to submit the result as a hexadecimal value, which confirmed that the final eax value from the trace is the flag formatted as picoCTF{0x...}.

   The return value in hex is the flag. Wrap it in picoCTF{...} if required, or submit it directly as a hex number.

   Learn more

   Reading x86 assembly is a fundamental reversing skill. Automated tools like Ghidra and IDA Pro decompile assembly to C-like pseudocode, but understanding the raw assembly allows you to verify and correct the decompiler output.

Flag

asm1(0x345): the argument 0x345 is compared against 0x37a, found smaller, so the function adds 3 and returns 0x348.