Magic Instruction prior to c function call

Support for Randall Hyde's "The Art of 64-Bit Assembly" book
Post Reply
olivs3268
Posts: 1
Joined: Tue Aug 22, 2023 1:17 pm

Magic Instruction prior to c function call

Post by olivs3268 »

Hello,

After assembling and linking the listing4-8 (page 202, "the art of 64 Assembly", vol1), i encounter an issue: the .exe file does not behave as expected.
However, if i change the "magic instructions" in the asmMain procedure code as follow:

Code: Select all

asmMain proc
    sub rsp, 56; instead of "sub rsp, 48" as in the book
; displays the fields of the string descriptor

    lea rcx, fmtStr
    mov edx, aString.maxLen ; zero extends!
    mov r8d, aString.len; zero extends
    mov r9, aString.strPtr
    call printf

    add rsp, 56; instead of "add rsp, 48" as in the book
    ret; returns to caller
asmMain endp
after assembling and linking the file with this version of listing4-8 asmMain, it prints correctly the output:
"Calling Listing 4-8:
aString: maxLen: 20, len: 20, string data:'Initial String Data'
Listing 4-8 terminated"

With the book version of asmMain, the output printed is only:
"Calling Listing 4-8:"

My machine is a Windows11 x64 bits laptop, I don't know the reason of this unexpected behaviour, and i encountered it on several listings examples of this book. I would be glad to discuss this problem (I surely missed something which could explain this, if it's the case i apologize for the inconvenience)
thanks in advance,

Olivier
rhyde
Site Admin
Posts: 51
Joined: Sun Dec 04, 2022 5:36 pm

Re: Magic Instruction prior to c function call

Post by rhyde »

Yes, this is a defect in the example (probably others, as well).
Before calling a function, the Microsoft ABI requires the stack to be 16-byte aligned (before the return address is pushed on the stack). As the return address adds 8 bytes to the stack, you must add some value to RSP that is n*16 + 8 if you are going to call any other API functions (which printf ultimately will). That's why 56 works (4*16 + 8) and 48 (4*16) does not.

Normally, upon entry into a function (especially asmMain), you would execute the standard entry sequence (see Chapter 5). This pushes RBP on the stack, so between the return address and RBP the stack is 16-bit aligned. In this case, sub rsp, 48 will work just fine because the stack is already 16-byte aligned.
Cheers,
Randy Hyde
Post Reply