winapi - How can I pass an F# delegate to a P/Invoke method expecting a function pointer?

Question

I'm attempting to set up a low-level keyboard hook using P/Invoke in an F# application. The Win32 function SetWindowsHookEx takes a HOOKPROC for its second argument, which I've represented as a delegate of (int * IntPtr * IntPtr) -> IntPtr, similar to how this would be handled in C#. When calling the method, I get a MarshalDirectiveException stating that the delegate parameter cannot be marshaled because

Generic types cannot be marshaled

I'm not sure how generics are involved, as all types are concretely specified. Can anyone shed some light on this? Code follows.

EDIT

This may have to do with the way the F# compiler deals with type signatures - Reflector indicates that the delegate LowLevelKeyboardProc is implemented as a method which accepts one argument of type Tuple<int, IntPtr, IntPtr> - and there would be the un-marshalable generic type. Is there a way around this somehow, or are F# functions simply not capable of being marshaled to native function pointers?

let WH_KEYBOARD_LL = 13

type LowLevelKeyboardProc = delegate of (int * IntPtr * IntPtr) -> IntPtr

[<DllImport("user32.dll")>]
extern IntPtr SetWindowsHookEx(int idhook, LowLevelKeyboardProc proc, IntPtr hMod, UInt32 threadId)

[<DllImport("kernel32.dll")>]
extern IntPtr GetModuleHandle(string lpModuleName)

let SetHook (proc: LowLevelKeyboardProc) =
    use curProc = Process.GetCurrentProcess ()
    use curMod = curProc.MainModule

    SetWindowsHookEx(WH_KEYBOARD_LL, proc, GetModuleHandle(curMod.ModuleName), 0u)

Answer 1

Your LowLevelKeyboardProc definition is wrong. Change from

type LowLevelKeyboardProc = delegate of (int * IntPtr * IntPtr) -> IntPtr

to

type LowLevelKeyboardProc = delegate of int * IntPtr * IntPtr -> IntPtr

or better yet

type LowLevelKeyboardProc = delegate of int * nativeint * nativeint -> nativeint

or better yet

[<StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc =
    delegate of int * nativeint * KBDLLHOOKSTRUCT -> nativeint

In all of the above cases, proc will need to use curried form rather than tupled form.

Also note that you should add SetLastError = true to all of the externed functions whose documentation says to call GetLastError upon failure (which is the case for GetModuleHandle, SetWindowsHookEx, and UnhookWindowsHookEx). That way if any fail (and you should be checking the return values...), you can simply raise a Win32Exception or call Marshal.GetLastWin32Error to get proper diagnostics.

EDIT: Just for the sake of clarity, here are all the P/Invoke signatures I successfully tested locally:

[<Literal>]
let WH_KEYBOARD_LL = 13

[<StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc = delegate of int * nativeint * KBDLLHOOKSTRUCT -> nativeint

[<DllImport("kernel32.dll")>]
extern uint32 GetCurrentThreadId()

[<DllImport("kernel32.dll", SetLastError = true)>]
extern nativeint GetModuleHandle(string lpModuleName)

[<DllImport("user32.dll", SetLastError = true)>]
extern bool UnhookWindowsHookEx(nativeint hhk)

[<DllImport("user32.dll", SetLastError = true)>]
extern nativeint SetWindowsHookEx(int idhook, LowLevelKeyboardProc proc, nativeint hMod, uint32 threadId)

Also note that this would work equally as well, if you prefer value semantics for KBDLLHOOKSTRUCT:

[<Struct; StructLayout(LayoutKind.Sequential)>]
type KBDLLHOOKSTRUCT =
    val vkCode      : uint32
    val scanCode    : uint32
    val flags       : uint32
    val time        : uint32
    val dwExtraInfo : nativeint

type LowLevelKeyboardProc = delegate of int * nativeint * byref<KBDLLHOOKSTRUCT> -> nativeint