Jul 26, 2012

Posted by | 0 Comments

Calling Convention – Part I (__cdecl)

Calling Convention – Part I (__cdecl)

Before reading, I assume that you must have sufficient knowledge of C/C++ & Assembly.

Introduction

During the process of learning C++ programming for Windows, you must have came across strange specifiers that sometime appear in front of function declarations, like __cdecl, __stdcall (WINAPI, CALLBACK), __fastcall, etc. These specifiers are called “Calling Conventions“.

What are the calling conventions?

When a function is called, the arguments are typically passed to it, and the return value is retrieved. A calling convention describes how the arguments are passed and values returned by functions. It also specifies how the function names are decorated.

Is it really necessary to understand the calling conventions to write good C programs?

No, not at all. However, it may be helpful with debugging. Also, it is necessary for linking C/C++ with assembly code.

How does it work?

No matter which calling convention is used, the following things will happen:

  1. All arguments are typically saved (pushed) on stack, but may also be in registers (__fastcall will be discussed in later post).
  2. Program execution jumps to the address of the called function.
  3. Inside the function, registers ESI, EDI, EBX, and EBP are saved on the stack. The part of code that performs these operations is called ‘Function Prologand usually is generated by the compiler.
  4.  The function-specific code is executed, and the return value is placed into the EAX register.
  5. Registers ESI, EDI, EBX, and EBP are restored from the stack. The piece of code that does this is called Function Epilog‘, and in most cases compiler generates it.
  6. Arguments are removed(popped) from stack. This operation is called stack cleanup and may be performed either inside the called function or by the caller, depending on the calling convention used.

As an example for the calling conventions (except for this (__thiscall)), we are going to use a simple function:

int Add( int nValue1, int nValue2 )
{
    return nValue1 + nValue2;
}

The call to this function will look like this:

int nResult = Add( 2, 3 );

Note: Remember to compile this example code as C. If you are compiling as C++ code use the example code below to avoid ‘C++ name decorations’ (C++ name decorations are beyond the scope of this post. Will be discussed in later posts). In this post I will explain ‘C  name decorations’.

#ifdef __cplusplus
extern "C" {
#endif
int Add( int nValue1, int nValue2 )
{
    return nValue1 + nValue2;
}
#ifdef __cplusplus
}
#endif

C calling convention ( __cdecl )

This convention is the default for C/C++ programs. If a project is set to use some other calling convention, we can still declare a function to use __cdecl:

int __cdecl Add( int nValue1, int nValue2 );

The main characteristics of __cdecl calling convention are:

  1. Arguments are passed from right to left, and placed on the stack.
  2. Stack cleanup is performed by the caller.

Function Name Decoration For cdecl 
Function name is decorated by prefixing it with an underscore character ‘_’.

_Add //underscore before function name

Now, take a look at an example of a __cdecl call:

; // push arguments to the stack, from right to left
push 3
push 2
; // call the function
call _Add
; // cleanup the stack by adding the size of the arguments to ESP register
add esp, 8
; // copy the return value from EAX to a local variable (int nResult)
mov DWORD PTR [nResult], eax

The called function is shown below:

; // function prolog
push ebp
mov ebp, esp
push ebx
push esi
push edi
; // return nValue1 + nValue2;
mov eax, DWORD PTR [nValue1]
add eax, DWORD PTR [nValue2]
; // function epilog
pop ebx
pop esi
pop edi
mov esp, ebp
pop ebp
ret

Conclusion

__cdecl is the default calling convention for C and C++ programs.

Advantage

The advantage of this calling convention is that it allows functions with a variable number of arguments to be used (e.g printf).

Disadvantage

The disadvantage is that it creates larger executables.

Cont. Calling Convention – Part II

Read More