c# - PInvokeStackImbalance exception when using IntPtr in .NET 4? (Works in .NET 3.5)

Question

Might be a bit of noob question but it's something that's been getting me in a pickle for the past few hours (or days)...

I'm calling a method from a DLL in my code in .NET Framework 4.0

  [DllImport("xeneth.dll")]
    public static extern ErrorCode XC_GetFrame(Int32 h, FrameType type, ulong ulFlags, IntPtr buff, uint size);

and then using it here:

if (XC_GetFrame(myCam, XC_GetFrameType(myCam), 0, IntPtr.Zero, (uint)fs) != ErrorCode.E_NO_FRAME)

However, when I run this in .NET 4.0 I get a P/INVOKE error, however... running this in 3.5 does not trigger this error. After myself and another programmer have gone over the code we seem to have put it down to the IntPtr running differently on 4.0.

My application needs to run in .NET 4.0 as a couple of features required by the application are only available in 4.0 ...

Is there anything that maybe i'm overlooking or have simply forgotten to include?

Any thoughts are much appreciated!

Tom

Update:

Native Declaration:

virtual ErrCode XCamera::GetFrame(FrameType type, unsigned long ulFlags, void *buffer, unsigned int size)

Error: A call to PInvoke function 'DLLTest!DLLTest.Form1::XC_GetFrameType' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

Answer 1

Two common causes for this:

1. Mismatch of calling conventions. Your C# code uses the default of stdcall. Perhaps the native code uses cdecl. Try adding CallingConvention=CallingConvention.Cdecl to the DllImport attribute.
2. Mismatch of parameter lists (see below).

Don't kid yourself that your code is alright because .net 3.5 doesn't raise this error. The error detection in .net 4 is better which is why you only see the errors there. But your code is broken in all .net versions.

---

The native definition for your C++ virtual method is:

virtual ErrCode XCamera::GetFrame(FrameType type, unsigned long ulFlags, 
    void * buffer, unsigned int size);

It looks like you are passing the object pointer as the first parameter in your pinvoke call. I think that could work, although I don't know enough about how virtual functions are handled when exported to know whether that's a problem. Presumably you have exported a plain C function to instantiate objects.

The other problem that I see is that on Windows, C/C++ long is 32 bits. A C# long is 64 bits. This means that the correct declaration for ulFlags is as uint in your C# code.