A Byte

If you're a beginner in reverse engineering, maybe you're in the right place. I'm going to explain this challenge for who doesn't understand anything about x64 assembly and how to convert the binary to C, and then, find the flag with python.

I'm using IDA so when I open this binary, this is what appears:

Ok, but what does that mean?

It means that we have the crt0 and main. crt0 is the first thing that the program does to initialize the stack, store initialized variables into the data section, clear uninitialized global and static variables into the bss section, and jump to the main function. Basically, we have any global variable saved in the stack and also a memory space to run the main. Let's focus on main.

push rbp  — push the memory address of the register base pointer (rbp) to the stack frame.

mov rbp, rsp  — copies the memory address of rsp to the top of the current stack frame. Now both rsp and rbp are equal and on the top of the stack.

sub rsp, 50h  — creates a space to store local variables. This space is 50h (hex) = 80 (decimal) = 'P' (ASCII). It goes on the top of the stack.

mov [rbp+var_44], edi  — puts the first 32-bit argument register (edi) into the offset var_44, which is argc (argument count).

mov [rbp+var_50], rsi  — puts the second 64-bit register (rsi) into var_50, which is argv[]. Using rsi (64-bit) makes sense because a vector needs more memory space.

mov rax, fs:28h  — stack canary, used to detect stack buffer overflows before malicious code can execute.

mov [rbp+var_8], rax  — stores the canary value into var_8.

xor eax, eax  — clears the register. Equal numbers xor'd always result in 0, so this is faster than a mov instruction. Uses 32-bit eax because it produces the same result as 64-bit rax with fewer bits.

cmp [rbp+var_44], 2  — compares argc with the value 2.

jz short loc_77B  — jumps to loc_77B if the comparison equals zero (i.e. they are equal). "short" means the jump is encoded as 2 bytes, distance range −128 to +127 bytes.

After this last instruction you can see two lines: red and green. Expanding it with var_44 = argc and var_50 = argv:

Red line path

nop  — the comparison didn't result in zero.

jmp short loc_765  — jumps to loc_765.

loc_765:  — loc_765 function.

lea rdi, s  — places the address of string "u do not know da wae" into rdi.

call _puts  — displays that message on screen.

mov eax, 0FFFFFFFFh  — copies −1 to eax. Uses as few bytes as possible to be cache-friendly.

jmp loc_891  — jumps to loc_891.

loc_891:

mov rsi, [rbp+var_8]  — copies var_8 (the canary) into rsi.

xor rsi, fs:28h  — clears the stack canary space.

jz short locret_8A5  — jumps to locret_8A5 if zero (no overflow detected).

If it goes red: call ___stack_chk_fail — aborts with a stack overflow message.

If it goes green: locret_8A5 — program finishes normally.

Green line path

Back at the end of main, following the green line:

loc_77B:

mov rax, [rbp+argv]  — rax stores argv's memory address.

mov rax, [rax+8]  — expands +8 bytes into the memory space.

mov [rbp+s], rax  — string "s" receives rax's value.

mov rax, [rbp+s]  — rax receives the value of string "s".

mov rdi, rax  — rdi receives rax's value.

call _strlen  — calculates the length of string "s".

mov [rbp+var_3C], eax  — stores eax into var_3C.

cmp [rbp+var_3C], 23h  — compares var_3C and 23h.

jnz short loc_761  — if not zero, jump to loc_761.

mov [rbp+var_40], 0  — stores 0 into var_40.

jmp short loc_7CD  — unconditional jump to loc_7CD.

loc_7CD:

mov eax, [rbp+var_40]  — eax receives var_40.

cmp eax, [rbp+var_3C]  — compares eax with var_3C.

jl short loc_7A5  — if condition met, jump to loc_7A5.

loc_7A5:

mov eax, [rbp+var_40]

movsxd rdx, eax  — sign-extends eax (32-bit) into rdx (64-bit), giving 4 more bytes of memory space.

mov rax, [rbp+s]

add rax, rdx

movzx ecx, byte ptr [rax]  — ecx receives 1 byte from rax; the remaining 24 bits are zero-extended.

mov eax, [rbp+var_40]

movsxd rdx, eax

mov rax, [rbp+s]

add rax, rdx

xor ecx, 1  — if equal → 0, else → 1.

mov edx, ecx

mov [rax], dl  — memory address of rax gets the value of 8-bit register dl.

add [rbp+var_40], 1  — var_40 + 1. Goes back to loc_7CD.

In brief, loc_7A5 is a for loop that increments until eax matches var_3C.

After loc_7CD follows the red line: a large block of mov instructions stores byte values (chars) into a string. At the end it jumps to loc_764 if not zero.

Pseudocode in C

Now we have enough context. Here is how IDA organized the pseudocode:

And we can clean it up further:

Just C code is not enough — we need to convert bytes to a string to get the flag. That's where Python comes in.

Flag

Also, this was just my approach. The challenge also provided a Python solution using the z3 (Microsoft) library:

Same flag: