c - linking dilemma (undefined reference) between MinGW and MSVC. MinGW fails MSVC works

Question

I'm trying to port an old C .dll library originally done with MSVC that uses BEA Tuxedo library to use MinGW.

I have encountered a situation where MSVC compiles and links one file but MinGW fails. The actual problem is in linking stage. There comes 'undefined reference' error.

Here's the minimal example to create a dll: (tpsetunsol_test.c)

#include <atmi.h>

void __stdcall msghandler(char *pszMessage, long lMessageLen, long lFlags)
{

}   

int Inittpsetunsol()
{           
    int ret = 0;

    tpsetunsol(msghandler);

    return ret;                     
}   

This compiles without errors:

gcc -Wall -fexceptions -g -O2 -DWIN32 -DNDEBUG -D_WINDOWS -ID:/dev/tuxedo/include   -o   tpsetunsol_test.o -c tpsetunsol_test.c

Here comes the error:

dllwrap --export-all-symbols -LD:/dev/tuxedo/lib -k --output-lib test.lib --output-def test.def --enable-stdcall-fixup --add-stdcall-alias -o IAWS.dll tpsetunsol_test.o -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lwtuxws32

C:MinGWindllwrap.exe: no export definition file provided.
Creating one, but that may not be what you want
tpsetunsol_test.o: In function `Inittpsetunsol':
d:devpsetunsol_test.c:13: undefined reference to `tpsetunsol'
collect2.exe: error: ld returned 1 exit status
C:MinGWindllwrap.exe: C:MinGWingcc exited with status 1

function declaration in atmi.h:

extern void (_TMDLLENTRY * _TMDLLENTRY tpsetunsol _((void (_TMDLLENTRY *)(char _TM_FAR *, long, long)))) _((char _TM_FAR *, long, long));

#define _TMDLLENTRY __stdcall
#define _TM_FAR

Version:

$ gcc -v
Using built-in specs.
COLLECT_GCC=C:MinGWingcc.exe
COLLECT_LTO_WRAPPER=c:/mingw/bin/../libexec/gcc/mingw32/4.7.2/lto-wrapper.exe
Target: mingw32
Configured with: ../gcc-4.7.2/configure --enable-    languages=c,c++,ada,fortran,objc,obj-c++ --disable-sjlj-exceptions --with-dwarf2 --enable-shared --enable-libgomp --disable-win32-registry --enable-libstdcxx-debug --disable-build-poststage1-with-cxx --enable-version-specific-runtime-libs     --build=ming
w32 --prefix=/mingw
Thread model: win32
gcc version 4.7.2 (GCC)

Edit: I found out using nm on object files created by MSVC and GCC that symbol tpsetunsol is different

Looking at _tpsetunsol symbol it is quite evident that MSVC and GCC produce different symbols.

GCC produces: U _tpsetunsol and MSVC: U _tpsetunsol@4

Edit: nm output after build with Haroogan's suggestion:

$ dllwrap --export-all-symbols -LD:/dev/tuxedo.64/lib --output-lib test.lib --output-def test.def -o IAWS.dll tpsetunsol_test.o -lwtuxws32_new
C:MinGWindllwrap.exe: no export definition file provided.
Creating one, but that may not be what you want
tpsetunsol_test.o: In function `Inittpsetunsol':
d:devIA/tpsetunsol_test.c:13: undefined reference to `tpsetunsol'
collect2.exe: error: ld returned 1 exit status
C:MinGWindllwrap.exe: C:MinGWingcc exited with status 1


$ nm tpsetunsol_test.o
00000000 b .bss
00000000 d .data
00000000 N .debug_abbrev
00000000 N .debug_aranges
00000000 N .debug_info
00000000 N .debug_line
00000000 N .debug_loc
00000000 r .eh_frame
00000000 t .text
00000004 T _Inittpsetunsol
00000000 T _msghandler@12
         U _tpsetunsol

$ nm ../tuxedo.64/lib/libwtuxws32.a  | grep -i tpsetuns
00000000 I __imp__tpsetunsol@4
00000000 T _tpsetunsol@4

output from gcc preprocessor (-E) (only the line tpsetunsol is declared)

extern void (__attribute__((__stdcall__)) * __attribute__((__stdcall__)) tpsetunsol (void (__attribute__((__stdcall__)) *)(char *, long, long))) (char *, long, long);

Answer 1

As you have already discovered yourself - name mangling may be different across compilers. To solve your problem follow the instructions:

1. Download and install (can build from source) gendef utility:

   • If you have usual MinGW (targeting 32-bit), then obtain it here;
   • If you have MinGW-w64 (targeting 64-bit), then obtain it here.

2. Run gendef wtuxws32.dll (will generate wtuxws32.def);

3. Run dlltool -D wtuxws32.dll -d wtuxws32.def -l libwtuxws32.a (will generate libwtuxws32.a);

4. Put libwtuxws32.a to D:/dev/tuxedo/lib;

5. Now link against it.