System-Wide Windows Hooks

 

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.

Windows allows many types of programmatic "hooks" to be set in order to intercept and modify information being sent to or from applications running on the system. The classes of information that can be hooked include Windows messages, keyboard and mouse events. Each hook set provides a callback function that the operating system will call whenever the requested event occurs.

Hooks can be installed for specific threads, either in the current or remote processes, or system-wide, which is essentially installing it for every thread in the system. There are important caveats to the types of remote processes whose threads can be hooked—these include those running in a different desktop, different window station, those in console applications (even if they have windows), and those running in high-protection or core system processes. Caveats aside, setting a system-wide hook often results in a deluge of information bombarding the hook callback functions in multiple threads in multiple process spaces, and therefore, to avoid affecting system performance, hook callback implementations should be thin and lightweight.

This article examines the correct usage of certain system-wide hooks on WinNT and Win9x platforms, looking at their APIs, idiosyncrasies, and side effects. Throughout the article, references to WinNT refer to the entire NT family—from NT 4 through Win2000, Win2003 and XP. The Win9x information is slowly becoming obsolete, as nearly all new Windows installations choose XP. However, a huge number of home users still are running Win9x, and any mainstream application intended for home users will need to take into account the large differences between the platforms regarding system-wide hooks.

Setting Hooks

Calling SetWindowsHookEx sets system-wide hooks, and a callback function with a generic footprint needs to be provided. For a full example of setting a system-wide hook, see HookListing1 at the end of this article. The function takes as parameters the type of hook being set, a pointer to the callback function of the hook, the instance handle of the dll containing the callback function, and a thread identifier. For system-wide hooks, the thread identifier is NULL. The return value is NULL if for some reason the hook could not be set, and otherwise the value is the handle of the newly set hook.

When the event being hooked occurs in the process space that set the hook, Windows simply passes the event to the callback function for handling before or after the intended target, depending on the nature of the hook. However, when the event being hooked occurs in a remote process space, Windows has to ensure that the code of the callback function exists in that process space. To allow this, Windows insists that the callback function of a system-wide hook be implemented inside a dll, which it maps into every process space in which the hook event occurs and needs to be handled. As a side effect, this mechanism becomes one of the few documented methods of injecting a dll into a remote process space. The injected dll has a reference count of 1, which is decremented once the hook is unset, allowing the dll to be unloaded. Additionally, before calling the callback function, the reference count is incremented again, and decremented upon exiting the callback function. This ensures that the dll will not be unmapped from the process space while the callback is being executed, if for example the hook were to be unset just then.

Unsetting Hooks

Hooks are unset by calling UnhookWindowsHookEx. (Again, for an example see the code at the end of this article.) The function is extremely simple, taking only the hook handle as its sole parameter. It returns a BOOL indicating whether the hook was successfully unset or not. Failure usually indicates that the hook has already been unset, or the handle otherwise invalid.

When a hook is unset, Windows decrements the reference count of every dll whose reference count was raised as a result of the hook being set. In principle, this should allow for the ejection of all the injected dlls. However, particularly on the NT platform, some processes may remain dormant for a long time, and the injected dll will remain loaded in the process memory space until something occurs to "jog" it.

Typically, although not reliably, triggering the event the hook is meant to intercept will cause Windows to check if a hook is set, realize that one is not, decrement the reference count, and cause the unloading of the dll from that process. Thus SendMessage can be used to jog and unload dlls loaded by send-message-type hooks, and PostMessage can be used to jog and unload dlls loaded by get-message-type hooks, keyboard events for keyboard hooks, and low-level keyboard events (such as ALT-CTRL-DEL) for low-level keyboard hooks. However, this jogging trick is not fully reliable, and injected dll "stickiness" is a serious consideration for applications that need to allow dynamic updating of dlls—since a dll loaded into some process space cannot be overwritten on the disc. Some processes even deliberately implement dll stickiness as an optimization—Windows Explorer is an example—since hanging on to frequently loaded and unloaded dlls can dramatically improve performance, by cutting out the considerable work done by the Windows Loader each time a dll has to be reloaded.

Due to the reference-count incrementing and decrementing mechanism, a certain design complexity arises with the unsetting of hooks. For example, it is sometimes required to set a system-wide hook during dll startup, and only unload it when the dll is unloaded. A naive implementation might well place the hook unsetting code in the DllMain of the hook dll, for the case of DLL_PROCESS_DETACH, so that when the dll is unloaded, it will unset the hook. However, since the dll has an incremented reference count, which will remain incremented as long as the hook is loaded, DllMain will never be called to unload, and the hook never unset. It is therefore necessary to design a more complex architecture that, external to the dll, knows when the hook is to be unset, and calls the unsetting code.

Despite being system wide, hooks remain a resource of the thread that set them. When that thread ends, all hooks set by it are automatically unset. While this allows good bookkeeping for the operating system, it can cause implementation complexity—for example, a thread that sets a hook cannot be allowed to terminate as long as the hook is needed.

Hook Callback Functions

Standard hook callback functions have the general format shown in the example, HookListing1.cpp. The first thing done is to check the value of the nCode parameter, to see if it is negative. If it is, we are advised by MSDN to do no processing other than to call the next hook in the chain, by calling CallNextHookEx. I found an explanation in an article written for Windows 3.1, "INFO: Importance of Calling DefHookProc ()" (MSDN Q74547), where one didn't explicitly call CallNextHookEx, but instead called DefHookProc. But I presume the principle is the same. They explain: "When the hook callback function receives a negative value for the nCode parameter, it should pass the message and the parameters to DefHookProc() without further processing. When nCode is negative, Windows is in the process of removing a hook callback function from the hook chain."

However, another MSDN article, "Win32 Hooks" by Kyle Marsh (MSDN 1993), has this to say:

In previous versions of Windows (before 3.1), the hook code indicated whether the filter function should process the event or call DefHookProc. If the hook code is less than zero, the filter function should not process the event; it should call DefHookProc, passing the three parameters it was passed without any modification. Windows used these negative codes to maintain the filter function chain, with help from the applications. Windows 3.1 still requires that if Windows sends a filter function a negative hook code, the filter function must call CallNextHookEx with the parameters Windows passed to the filter function. The filter function must also return the value returned by CallNextHookEx. Windows 3.1 never sends negative hook codes to filter functions.

I have never encountered a negative value for nCode, either Win9x or WinNT, but nevertheless follow the MSDN advice.

In the body of the function, one can access extensive hook-specific data. Some hooks, such as WH_GETMESSAGE hook, allow the data to be changed. This can give immense power to the hook-setter to alter the internal behaviour of remote processes. Other hooks, such as WH_CALLWNDPROC hook, allow it to be examined, but not changed. This is probably due to the synchronous nature of sent messages as opposed to posted ones—altering the data could conceivably cause the application to crash. In some hooks, such as WH_CALLWNDPROCRET, which is called when the window procedure has completed handling the sent message, changing the data has no meaning, since the event being hooked has already been fully handled.

Debugging a system-wide hook callback function can be complicated, since the dll is typically injected into multiple process spaces, and the quantity of callback events can be copious. SoftIce, marketed by Compuware, allows multiple processes to be debugged simultaneously, as well as stopping the system clock while debugging. It is also highly recommended that the dll with the callback function be rebased to some address not already used by any dll in any running process space. This ensures that breakpoints and debugging symbols can be easily handled in all process spaces into which the dll is injected.

Daisy Chaining

The last part of the callback function is passing on the event to the next hook in the chain, if there happens to be one, by calling CallNextHookEx. If one fails to do this, most hook events stop passing down the calling chain, and pass over any other hooks. However, the event will still reach its intended destination—just no further hooks will be aware of it. It is generally poor programming practice to interfere with other hooks, although I have encountered it in well-known mainstream applications.

As well as the parameters passed into the callback function itself, this function takes the handle to the current hook. This is a very problematic requirement, since the callback does not itself get passed the handle of the current hook as a parameter, and has no true way of knowing it. The solution is to store the hook handle in global memory, but this fails with system-wide hooks, where when dlls are injected into different process spaces—in a remote process space, the global variable is not set, and the only solution is to place the hook handle inside shared memory. An example is shown in HookListing1.cpp.

It turns out that NT is very forgiving of the requirement to pass the current hook handle when daisy-chaining—in fact I have disassembled CallNextHookEx on NT, and no reference at all is made to this parameter. However Win9x is not forgiving, and uses it to find the next hook in the chain. If the correct handle is passed on 9x, the event is correctly daisy-chained. If a NULL handle is passed, the event is not daisy-chained. If an incorrect current hook handle is passed the system is liable to crash.

If several identical hooks are set (e.g. from within different executables using the same hook dll, or different instances of the same application running simultaneously), with the same callback function, then difficult if not impossible "accounting" is needed to ensure the correct handle is always passed on. The best solution for Win9x is to only set a single instance of a given hook, with some method of alerting the other application instances to reset the hook if the initial one is unset.

No official guarantee is given regarding the order of calling hook callbacks. However, I have found that in general, the chain is traversed in reverse order, with the last hook set getting called first. Thus, if a rogue hook which fails to daisy-chain is set once your hook is already set, your hook will receive no events, and nor will you get any notification that your hook is essentially useless.

Notes on Specific Hooks

It is important to understand the context of an event when examining hooks, since hook callbacks are called in a specific context—a process and thread related to the hook. Some hooks are called in the context where the hook was set, while others are called in the context where one of several specific hook-related events occurs.

CallWndProc Hook

The CallWndProc hook callback function is only called for messages sent with SendMessage, not PostMessage.

On Windows 9x and Windows NT 3.51, the system calls this function whenever the thread calls the SendMessage function. The callback is called in the context of the thread that calls SendMessage, not the thread that receives the message. This means that the callback is called immediately upon the message being sent, irrespective of it being received. The callback is called in the context of the sender, and so the dll with the callback function will be mapped into the process space of the sender.

On Windows NT 4.0 and later, the system calls this function before calling the window procedure to process a message sent to the thread. This means that the callback is called when the message is received, just before the Window Proc processes it. The callback is called in the context of the receiver, and so the dll containing the callback function will be mapped into the process space of the receiver. Since this hook is called synchronously when a message is sent, the callback function should be very cautious about itself sending messages to either the target window, or other windows, since endless recursion can occur. In particular, some Windows and particularly MFC functions themselves send window messages, allowing this recursion to occur implicitly.

CallWndProcRet Hook

The CallWndProcRet hook callback function is only called for messages sent with SendMessage, not PostMessage. It is called after the window procedure has handled the message, and has returned. Thus it serves as a complement to the CallWndProc hook, letting one know that an even has occurred, and already been handled.

In which context is the callback called? There is no mention in the documentation of the answer to this question, but I have found that NT behaves differently to 95/98. In fact, the same rules seem to apply here as apply to the CallWndProc hook, where the documentation does state where the callback is called.

GetMessage Hook

The system calls this function whenever the GetMessage or PeekMessage function has retrieved a message from an application message queue. Before returning the retrieved message to the caller, the system passes the message to the hook procedure. This means that the hook callback is called for messages posted to the window, not sent, or any other way of getting to the window bypassing the message queue. The hook callback is called not when the message arrives in the message queue, but when it is pulled off of it. In many ways, therefore, this hook is the complement to the CallWndProc hook—while that hook deals with sent messages, this one deals with posted messages.

The hook callback is called in the context of the receiver of the message, not its sender, and so the dll containing the callback function will be mapped into the process space of the receiver. On Win9x using this hook, and then posting a broadcast message, is the only way, using hooks, to get a dll injected into every process with a message loop, since both CallWndProc and CallWndProcRet hook callbacks on Win9x are called in the context of the sender of the message.

Debug Hook

This hook was designed to allow debugging of any other hook. In principle, therefore, by not passing the hook event further down the chain, it is possible to use it to prevent any hook in the system working. I have found however that both LowLevelKeyboard and LowLevelMouse events cannot be prevented from reaching their hooks using a Debug hook.

Although the MSDN documentation is unclear, the callback is called in the context of the callback it is intercepting. This means that the dll with the callback code gets mapped into the process space of any process whose hook callback gets called.

Keyboard Hook

This hook exists on 9x and NT platforms, and unlike the LowLevelKeyboard, causes dll injection, with the callback being called in multiple processes, not just the one which set the hook. Setting the hook causes almost instantaneous injection into multiple processes, suggesting that the keyboard event is placed in the message queues of many processes, and not just that of the active process. This has severe ramifications for Win9x, regarding the hook handle being placed in shared memory, and not allowing multiple settings of the same Keyboard hook by the application, as described above.

Low Level Keyboard Hook

This hook exists in the NT family only, and is able to intercept extra keyboard events that the keyboard hook is not able to detect. It is called in the context of the process that set the hook, and as such, does not cause the dll to be injected into other process spaces. Nevertheless, when the hook is set, Windows will increment the dll reference count.

The system requires that the hook callback return very quickly. The speed is undefined, unless a certain registry value is defined. However, I have found empirically that the system readily generates another low-level keyboard event that bypasses the hook if the callback takes too long. The best treatment is to trigger a system event, and return immediately, and have a different thread wait upon the system event.

The context in which each hook calls its callback is summarized in this table:

Hook TypeCallWndProcCallWndProcRetGetMessageDebugKeyboardLow Level KeyboardCallback ContextWin9x: SendMessage sender
WinNT: SendMessage receiverWin9x: SendMessage sender
WinNT: SendMessage receiverReceiver of PostMessage messageContext of hook callback being interceptedProcess which receives keyboard eventProcess which sets hook

I have already described why setting certain system-wide hooks causes dll injection into remote processes. However, to use a hook as a vehicle for dll injection is highly problematic for many reasons. For one, there is no fine control. One often will have the dll injected unnecessarily into many process spaces, with the attendant performance hit. Furthermore, many processes for which system-wide hooks do not work are likewise impervious to this method of injection. Also, there are significant differences between Win9x and WinNT regarding each type of hook, meaning that code intended to be cross-platform often has to have per-platform treatment. And the individual idiosyncrasies of the various hooks, few of which are clearly documented, mean that extensive debugging and experimentation is needed before injection can be relied upon.

Despite these caveats, widespread injection via a system-wide hook can be achieved on Win9x by setting a GetMessage hook and posting a broadcast message, and on WinNT by setting a CallWndProc or CallWndProcRet hook, and sending a broadcast message. Using the value HWND_BROADCAST instead of a window handle in the SendMessage or PostMessage call sends broadcast messages. To avoid a deadlock when sending a broadcast message, it is better to use the SendMessageTimeout function, which ensures that no frozen application can infinitely delay the propagation of the message.

The following code shows an implementation of a system-wide hook.

Example: HookListing1.cpp

#include #include "HookListing1.h"///////////////////////////////////////////////////// Save hook handle in shared data section, so that// it is visible in all process spaces - an absolute// requirement on Win9x// This code is the standard way to save data in a shared// data segment. Note both the compiler and linker directives,// the short name of the data segment, and that the variable// must be initialized.#pragma data_seg( ".SHARED" )HHOOK g_hCurrentHook = NULL;#pragma data_seg()#pragma comment(linker, "/SECTION:.SHARED,RWS") ///////////////////////////////////////////////////HINSTANCE g_hInstance = NULL;BOOL APIENTRY DllMain( HANDLE hModule,                        DWORD  ul_reason_for_call,                        LPVOID ){    switch (ul_reason_for_call){case DLL_PROCESS_ATTACH:g_hInstance = (HINSTANCE)hModule;case DLL_THREAD_ATTACH:case DLL_THREAD_DETACH:case DLL_PROCESS_DETACH:break;    }    return TRUE;}//// skeletal hook callback function// Typically, you need a separate callback for each type of hook so// that you know how to cast the lParam to a hook-specific structure// Note that we do not need to worry about the hook being unset or// the hook dll being ejected while this function is executing//LRESULT CALLBACK HookProc(  int nCode,      // hook code  WPARAM wParam,  // current-process flag  LPARAM lParam   // message data){if(nCode>=0){// examine the current process flag, if needed// cast lParam to a hook specific structure, and// do something with it}return CallNextHookEx(  g_hCurrentHook,// handle to current hook  nCode,// hook code passed to hook procedure  wParam,// value passed to hook procedure  lParam);// value passed to hook procedure}//// bare-bone hook setting function. // Makes no check for the hook already existing// Returns true if hook was successfully set//HOOKDLL_API bool SetHook(){g_hCurrentHook = SetWindowsHookEx(WH_CALLWNDPROC,// hook typeHookProc,// hook procedureg_hInstance,// handle to application instance0   );// thread identifierif(NULL==g_hCurrentHook){// use GetLastError() to find out why it failedreturn false;}return true;}//// bare-bone hook unsetting function. // Returns true if hook was successfully unset, // or if hook was not previously set//HOOKDLL_API bool UnsetHook(){if(g_hCurrentHook){if(!UnhookWindowsHookEx(g_hCurrentHook)){// use GetLastError() to find out why it failedreturn false;}}g_hCurrentHook = NULL;return true;}

---

Moishe is a Windows Internal programmer, currently working for Mercury Interactive. You can contact him at [e-mail]

Contact BYTE.com at http://www.byte.com/feedback/feedback.html.