c++ - SetWindowsHookEx hooking into every running program

Question

I have written a small DLL that is being injected into a game to apply runtime fixes. I've decided to add some Keyboard Listeners:

while (1)
    if (AsyncKeyState(...)) (...)

However, this is taking up CPU usage, and introducing some noticeable stuttering. So I decided to use a callback methodology as suggested by MSDN via SetWindowsHookExA and a KeyboardProc callback. At first, I was running the method via SetWindowsHookExA(WH_Keyboard, KeyboardProc, NULL, 0) but was getting error code ERROR_HOOK_NEEDS_HMOD (0x594). So I changed my code to become like this:

bool APIENTRY DllMain(HMODULE hModule, DWORD reason, LPVOID lpRes)
{
    HHOOK hook_ = nullptr;
    switch (reason)
    {
    case DLL_PROCESS_ATTACH:
        
        if (!(hook_ = SetWindowsHookEx(WH_KEYBOARD, KeyboardProc, hModule, 0)))
        {
            char x[100];
            sprintf(x, "Failed To Hook Keyboard FN: 0x%X", GetLastError());
            MessageBox(NULL, x, "Error", MB_ICONERROR);
        }
        break;
    case DLL_THREAD_ATTACH:
    case DLL_THREAD_DETACH:
    case DLL_PROCESS_DETACH:
        UnhookWindowsHookEx(hook_);
        break;
    }
    return true;
}

This was not giving the error anymore, and my hooking Keyboard function was working perfectly. However, a very unintended side-effect was that all keyboard captures from all running applications are being ignored. I noticed this quote from MSDN, so I added it to my code:

If code is greater than or equal to zero, and the hook procedure did not process the message, it is highly recommended that you call CallNextHookEx and return the value it returns; otherwise, other applications that have installed WH_KEYBOARD hooks will not receive hook notifications and may behave incorrectly as a result. If the hook procedure processed the message, it may return a nonzero value to prevent the system from passing the message to the rest of the hook chain or the target window procedure.

That fixed the keyboard issue, but now there's another issue, which is I'm unable to delete the DLL I created. When I attempt to delete it, it says "dll is running in Explorer.exe". when I kill explorer.exe, it says "dll is running in SteamHelper.exe". When I kill steamhelper.exe, it says "dll is running in Chrome.exe". This shows me that I've somehow hooked into every running application?

I'm unable to fix this issue, and I couldn't find any help on it. Any insight is greatly appreciated!

question from:https://stackoverflow.com/questions/65922595/setwindowshookex-hooking-into-every-running-program

Answer 1

You ARE hooking every application process, because you are installing the hook globally, by setting the dwThreadId parameter of SetWindowsHookEx() to 0:

dwThreadId

Type: DWORD

The identifier of the thread with which the hook procedure is to be associated. For desktop apps, if this parameter is zero, the hook procedure is associated with all existing threads running in the same desktop as the calling thread. For Windows Store apps, see the Remarks section.

Instead, you should set that parameter to the actual thread ID of the game thread that you want to hook events for.