macos - Code segfaults unless esp incremented

Question

I have the following code all it is suppose to do is print Hello World using CoreFoundation functions. However whenever I seemingly have a proper aligned stack it doesn't work, seg faulting. But then when I finally got it working the stack isn't aligned?!?!?!

global  _main

align 4, db 0x90

extern _CFStringCreateWithCString
extern _CFShow

section .data
    hw:  db 'Hello World!' ,0xA,0

section .text
    _main: ; entering a new function stack must be balanced right?

    push ebp ; saving ebp  (esp + 4)
    mov  ebp, esp ; moving registers around 
    ; align stack as calling pushed a 4 byte address on to the stack
    sub  esp, 12 ; balancing the stack back to mod 16 (4 + 12 = 16)


    push 8   ; 4 bytes
    push hw  ; 4 bytes
    push 0   ; 4 bytes
    call _CFStringCreateWithCString ; 4 bytes

    ; stack still balanced

    sub  esp, 12 ; 12 bytes
    push eax     ; 4 bytes
    call _CFShow ; 4 bytes

    ; that is 20 bytes?!?!? yet when I change the 12 to an 8 it doesn't run and instead segfaults! When I have the stack balanced!


    mov eax, 99 ; return value

    mov esp, ebp ; restore stack for function that called us
    pop ebp
    ret          ; return

When run it works, however I can find no reason why it does. I have to subtract 12 from esp for a one argument function. Shouldn't it be 8, doesn't push already handle incrementing the stack for the argument?

Answer 1

I'm not sure why the original function is doing additional subtractions from the stack without using the space which is so allocated on the stack. The stack grows down on the x86. In this context, if you do:

sub esp, NUMBER

you are allocating (making available) NUMBER bytes on the stack to be used for some purpose.

I'm making an assumption that the library follows a C calling convention:

1) Push the parameters (in reverse order) onto the stack
2) Call the function
3) Restore the stack based upon the amount of space used by the prior pushes.

With these things in mind, here's how I'd write your function:

global  _main

align   4, db 0x90

extern _CFStringCreateWithCString
extern _CFShow

section .data
hw: db 'Hello World!' ,0xA,0

section .text
_main: ; entering a new function stack must be balanced right?

    push   ebp         ; saving ebp  (esp + 4)
    mov    ebp, esp    ; set stack frame pointer 

    push   8           ; String encoding - 4 bytes
    push   hw          ; String pointer - 4 bytes
    push   0           ; Allocator [0 for default] - 4 bytes
    call   _CFStringCreateWithCString
    add    esp, 12     ; restore the stack [pop the 12 bytes back off]

    push   eax         ; Address of string to show (returned by prior call) - 4 bytes
    call   _CFShow
    add    esp, 4      ; restore the stack [pop the 4 bytes back off] NOT NEEDED with 

    mov    eax, 99     ; return value

    mov    esp, ebp    ; restore stack for function that called us
    pop    ebp
    ret

Note that since the last mov instruction restores the stack, then the last add esp,4 could be omitted, but it's here for completeness.

---

MacOS requires / guarantees 16-byte alignment of the stack pointer for function calls. To do this:

global  _main

align   4, db 0x90

extern _CFStringCreateWithCString
extern _CFShow

section .data
hw: db 'Hello World!' ,0xA,0

section .text
_main:
 ; ESP was aligned before the call instruction pushed a return address
 ; now the nearest alignment boundaries are ESP+4 and ESP-12

    push   ebp         ; saving ebp  (esp + 4)
    mov    ebp, esp    ; set stack frame pointer 

 ; ESP-8 is 16-byte aligned; not enough room for 12 bytes of args
    sub    esp,12      ; So we have to go past that to aim for the *next* alignment boundary
    push   8           ; String encoding - 4 bytes
    push   hw          ; String pointer - 4 bytes
    push   0           ; Allocator [0 for default] - 4 bytes
    call   _CFStringCreateWithCString

    ;add    esp, 12+12 - 4    ; pop the padding and args, then sub 4 for 16-byte alignment on next call (after push)
    ;push   eax         ; Address of string to show (returned by prior call) - 4 bytes

    mov    [esp], eax   ; reuse the stack reservation; ESP is still aligned
    call   _CFShow
    add    esp, 12+12   ; restore the stack [pop the args + padding back off]

    mov    eax, 99     ; return value

    mov    esp, ebp    ; restore stack for function that called us
    pop    ebp
    ret

Likewise as in the first case, the last add esp,24 could be omitted.