Calling Conventions Demystified (Visual C++ calling conventions explained)

Long time no see these tedious 0CCCCCCCCh, "0cch" machine code for int 3, just a way to initialize the stack with int 3 for debug version:)

1. #   lea         edi,[ebp-0C0h] 
2. #   mov         ecx,30h 
3. #   mov         eax,0CCCCCCCCh 
4. #   rep stos    dword ptr [edi] 

Calling Conventions Demystified

By Nemanja Trifunovic

Visual C++ calling conventions explained

Introduction

During the long, hard, but yet beautiful process of learning C++programming for Windows, you have probably been curious about thestrange specifiers that sometime appear in front of functiondeclarations, like __cdecl, __stdcall, __fastcall, WINAPI, etc. After looking through MSDN, or some other reference, you probably found out that these specifiers specify the calling conventionsfor functions. In this article, I will try to explain different callingconventions used by Visual C++ (and probably other Windows C/C++compilers). I emphasize that above mentioned specifiers areMicrosoft-specific, and that you should not use them if you want towrite portable code.

So, what are the calling conventions? When a function is called, thearguments are typically passed to it, and the return value isretrieved. A calling convention describes how the argumentsare passed and values returned by functions. It also specifies how thefunction names are decorated. Is it really necessary to understand thecalling conventions to write good C/C++ programs? Not at all. However,it may be helpful with debugging. Also, it is necessary for linkingC/C++ with assembly code.

To understand this article, you will need to have some very basic knowledge of assembly programming.

No matter which calling convention is used, the following things will happen:

1. All arguments are widened to 4 bytes (on Win32, of course), and putinto appropriate memory locations. These locations are typically on thestack, but may also be in registers; this is specified by callingconventions.
2. Program execution jumps to the address of the called function.
3. Inside the function, registers ESI, EDI, EBX, and EBP are saved onthe stack. The part of code that performs these operations is called function prolog and usually is generated by the compiler.
4. The function-specific code is executed, and the return value is placed into the EAX register.
5. Registers ESI, EDI, EBX, and EBP are restored from the stack. The piece of code that does this is called function epilog, and as with the function prolog, in most cases the compiler generates it.
6. Arguments are removed from the stack. This operation is called stack cleanup and may be performed either inside the called function or by the caller, depending on the calling convention used.

As an example for the calling conventions (except for this), we are going to use a simple function:

1. int sumExample (int a, int b)
2. {
3.     return a + b;
4. }

The call to this function will look like this:

1. int c = sum (2, 3);

For __cdecl, __stdcall, and __fastcall callingconventions, I compiled the example code as C (not C++). The functionname decorations, mentioned later in the article, apply to the Cdecoration schema. C++ name decorations are beyond the scope of thisarticle.

C calling convention (__cdecl)

This convention is the default for C/C++ programs (compiler option/Gd). If a project is set to use some other calling convention, we canstill declare a function to use __cdecl:

1. int __cdecl sumExample (int a, int b);

The main characteristics of __cdecl calling convention are:

1. Arguments are passed from right to left, and placed on the stack.
2. Stack cleanup is performed by the caller.
3. Function name is decorated by prefixing it with an underscore character '_' .

Now, take a look at an example of a __cdecl call:

1. ; // push arguments to the stack, from right to left
2. push        3    
3. push        2    
4. 
   
5. ; // call the function
6. call        _sumExample 
7. 
   
8. ; // cleanup the stack by adding the size of the arguments to ESP register
9. add         esp,8 
10. 
    
11. ; // copy the return value from EAX to a local variable (int c)
12. mov         dword ptr [c],eax

The called function is shown below:

1. ; // function prolog
2.   push        ebp  
3.   mov         ebp,esp 
4.   sub         esp,0C0h 
5.   push        ebx  
6.   push        esi  
7.   push        edi  
8.   lea         edi,[ebp-0C0h] 
9.   mov         ecx,30h 
10.   mov         eax,0CCCCCCCCh 
11.   rep stos    dword ptr [edi] 
12.   
13. ; //    return a + b;
14.   mov         eax,dword ptr [a] 
15.   add         eax,dword ptr [b] 
16. 
    
17. ; // function epilog
18.   pop         edi  
19.   pop         esi  
20.   pop         ebx  
21.   mov         esp,ebp 
22.   pop         ebp  
23.   ret

Standard calling convention (__stdcall)

This convention is usually used to call Win32 API functions. In fact, WINAPI is nothing but another name for __stdcall:

1. #define WINAPI __stdcall

We can explicitly declare a function to use the __stdcall convention:

1. int __stdcall sumExample (int a, int b);

Also, we can use the compiler option /Gz to specify __stdcall for all functions not explicitly declared with some other calling convention.

The main characteristics of __stdcall calling convention are:

1. Arguments are passed from right to left, and placed on the stack.
2. Stack cleanup is performed by the called function.
3. Function name is decorated by prepending an underscore characterand appending a '@' character and the number of bytes of stack spacerequired.

The example follows:

1. ; // push arguments to the stack, from right to left
2.   push        3    
3.   push        2    
4.   
5. ; // call the function
6.   call        _sumExample@8
8. ; // copy the return value from EAX to a local variable (int c)  
9.   mov         dword ptr [c],eax

The function code is shown below:

1. ; // function prolog goes here (the same code as in the __cdecl example)
2. 
   
3. ; //    return a + b;
4.   mov         eax,dword ptr [a] 
5.   add         eax,dword ptr [b] 
6. 
   
7. ; // function epilog goes here (the same code as in the __cdecl example)
8. 
   
9. ; // cleanup the stack and return
10.   ret         8

Because the stack is cleaned by the called function, the __stdcall calling convention creates smaller executables than __cdecl,in which the code for stack cleanup must be generated for each functioncall. On the other hand, functions with the variable number ofarguments (like printf()) must use __cdecl,because only the caller knows the number of arguments in each functioncall; therefore only the caller can perform the stack cleanup.

Fast calling convention (__fastcall)

Fast calling convention indicates that the arguments should beplaced in registers, rather than on the stack, whenever possible. Thisreduces the cost of a function call, because operations with registersare faster than with the stack.

We can explicitly declare a function to use the __fastcall convention as shown:

1. int __fastcall sumExample (int a, int b);

We can also use the compiler option /Gr to specify __fastcall for all functions not explicitly declared with some other calling convention.

The main characteristics of __fastcall calling convention are:

1. The first two function arguments that require 32 bits or less areplaced into registers ECX and EDX. The rest of them are pushed on thestack from right to left.
2. Arguments are popped from the stack by the called function.
3. Function name is decorated by by prepending a '@' character andappending a '@' and the number of bytes (decimal) of space required bythe arguments.

Note: Microsoft have reserved the right to change the registers for passing the arguments in future compiler versions.

Here goes an example:

1. ; // put the arguments in the registers EDX and ECX
2.   mov         edx,3 
3.   mov         ecx,2 
4.   
5. ; // call the function
6.   call        @fastcallSum@8
7.   
8. ; // copy the return value from EAX to a local variable (int c)  
9.   mov         dword ptr [c],eax

Function code:

1. ; // function prolog
3.   push        ebp  
4.   mov         ebp,esp 
5.   sub         esp,0D8h 
6.   push        ebx  
7.   push        esi  
8.   push        edi  
9.   push        ecx  
10.   lea         edi,[ebp-0D8h] 
11.   mov         ecx,36h 
12.   mov         eax,0CCCCCCCCh 
13.   rep stos    dword ptr [edi] 
14.   pop         ecx  
15.   mov         dword ptr [ebp-14h],edx 
16.   mov         dword ptr [ebp-8],ecx 
17. ; // return a + b;
18.   mov         eax,dword ptr [a] 
19.   add         eax,dword ptr [b] 
20. ;// function epilog  
21.   pop         edi  
22.   pop         esi  
23.   pop         ebx  
24.   mov         esp,ebp 
25.   pop         ebp  
26.   ret

How fast is this calling convention, comparing to __cdecl and __stdcall? Find out for yourselves. Set the compiler option /Gr, and compare the execution time. I didn't find __fastcall to be any faster than other calling conventons, but you may come to different conclusions.

Thiscall

Thiscall is the default calling convention for callingmember functions of C++ classes (except for those with a variablenumber of arguments).

The main characteristics of thiscall calling convention are:

1. Arguments are passed from right to left, and placed on the stack. this is placed in ECX.
2. Stack cleanup is performed by the called function.

The example for this calling convention had to be a littledifferent. First, the code is compiled as C++, and not C. Second, wehave a struct with a member function, instead of a global function.

1. struct CSum
2. {
3.     int sum ( int a, int b) {return a+b;}
4. };

The assembly code for the function call looks like this:

1.   push        3    
2.   push        2    
3.   lea         ecx,[sumObj] 
4.   call        ?sum@CSum@@QAEHHH@Z            ; CSum::sum 
5.   mov         dword ptr [s4],eax

The function itself is given below:

1.   push        ebp  
2.   mov         ebp,esp 
3.   sub         esp,0CCh 
4.   push        ebx  
5.   push        esi  
6.   push        edi  
7.   push        ecx  
8.   lea         edi,[ebp-0CCh] 
9.   mov         ecx,33h 
10.   mov         eax,0CCCCCCCCh 
11.   rep stos    dword ptr [edi] 
12.   pop         ecx  
13.   mov         dword ptr [ebp-8],ecx 
14.   mov         eax,dword ptr [a] 
15.   add         eax,dword ptr [b] 
16.   pop         edi  
17.   pop         esi  
18.   pop         ebx  
19.   mov         esp,ebp 
20.   pop         ebp  
21.   ret         8

Now, what happens if we have a member function with a variable number of arguments? In that case, __cdecl is used, and this is pushed onto the stack last.

Conclusion

To cut a long story short, we'll outline the main differences between the calling conventions:

• __cdecl is thedefault calling convention for C and C++ programs. The advantage ofthis calling convetion is that it allows functions with a variablenumber of arguments to be used. The disadvantage is that it createslarger executables.
• __stdcall is used to call Win32 API functions. It does not allow functions to have a variable number of arguments.
• __fastcall attempts to put arguments in registers, rather than on the stack, thus making function calls faster.
• Thiscall calling convention is the default calling convention used by C++ member functions that do not use variable arguments.

In most cases, this is all you'll ever need to know about the calling conventions.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Nemanja Trifunovic


Born in Kragujevac, Serbia, and lived there until 2000. Spent fourwonderful years in San Diego, California, one year in Boston, one inNew York and now lives again in Boston area with his wife and twolittle daughters.

Wrotehis first program in 1984, became a professional developer after hegraduated in 1994. After all these years still very passionate aboutprogramming and software development in general.
Occupation: Software DeveloperLocation: United States