asm1 picoCTF 2019 Solution

Description

1. Step 1Read the assembly and set up the stack frame
   Open test.S. The function asm1 takes one argument (0x6fa = 1786). Trace through it manually: set up the stack frame in your head (or on paper), tracking the value of eax and other registers at each instruction.
   bash

   cat test.S

   Copied!
   Learn more

   In x86 calling convention (cdecl), arguments are pushed on the stack right-to-left. The first argument is at [ebp+8] after the function prologue (push ebp; mov ebp, esp). The return value is in eax when the function returns.

   Key x86 instructions: cmp a, b sets flags based on a - b. jg jumps if greater (signed). jl jumps if less. je jumps if equal. add and sub modify a register. mov copies a value.

2. Step 2Trace the function logic
   Start with the argument value 0x6fa in [ebp+8]. Follow each branch condition (compare the argument to hardcoded values) to determine which branch is taken. Track the final value loaded into eax before ret.
   Learn more

   Compile the assembly and run it to verify your answer: gcc -m32 -o test test.S -no-pie && python3 -c "import ctypes; lib=ctypes.CDLL('./test'); print(hex(lib.asm1(0x6fa)))". This is often faster than manual tracing for complex functions.

3. Step 3Submit the return value as the flag
   The return value in hex is the flag. Wrap it in picoCTF{...} if required, or submit it directly as a hex number.
   Learn more

   Reading x86 assembly is a fundamental reversing skill. Automated tools like Ghidra and IDA Pro decompile assembly to C-like pseudocode, but understanding the raw assembly allows you to verify and correct the decompiler output.

Flag

asm1(0x345): the argument 0x345 is compared against 0x37a, found smaller, so the function adds 3 and returns 0x348.