python - Shared library dependencies with distutils

Question

I'm a newbie to distutils and I have a problem that really has me stuck. I am compiling a package that requires an extension, so I make the extension thus:

    a_module = Extension(
          "amodule",
          ["initmodule.cpp"],
          library_dirs=libdirs,
          extra_objects = [
                    "unix/x86_64/lib/liba.so"
                    "unix/x86_64/lib/lib.so",
                    "unix/x86_64/lib/libc.so"],
    )

I then run the setup method:

    setup(name="apackage", version="7.2",
      package_dir = {'':instdir+'/a/b/python'},
      packages=['apackage','package.tests'],
      ext_modules=[hoc_module]
)

The package distribution is made properly and I can "python setup.py install" fine but when I try and import my package I get an error ImportError: liba.so.0: cannot open shared object file: No such file or directory

I realise that when I add the location of liba.so.0 to my LD_LIBRARY_PATH the program runs fine. Unfortunately I haven't written these modules and don't have a good understanding of compilation. I've been trying to figure this out for several days to no avail.

UPDATE:I tried passing the liba.a, libb.a etc files to extra_objects but this didn't work, generating the following errror: liba.a: could not read symbols: Bad value collect2: ld returned 1 exit status. What I'm trying to do is package a python module which requires a library to be compiled which itself depends on other libraries which I need to somehow include in the package .I suspect that my problem is very similar to this one: http://mail.python.org/pipermail/distutils-sig/2009-February/010960.html but that one was not resolved, I thought perhaps since it's two years old a resolution has been found?

UPDATE 2: For now I have solved this by doing:

      data_files=[('/usr/local/lib', glob.glob('unix/x86_64/lib/*'))]

That is to say, I am copying the libraries I need into /usr/local/lib. I'm not hugely happy with this solution however, not least because it requires my users to have root privileges and also because this may still not work Redhat distros. So if anyone can suggest something better than this fix do please let me know.

Answer 1

You can have the linker store paths to search in the output binary so LD_LIBRARY_PATH isn't necessary. Some examples:

# Will link fine but at run-time LD_LIBRARY_PATH would be required
gcc -o blah blah.o -lpcap -L/opt/csw/lib

# Without LD_LIBRARY_PATH=/opt/csw/lib it will fail to link, but
# it wouldn't be needed at run-time
gcc -o blah blah.o -lpcap -Wl,-R/opt/csw/lib

# LD_LIBRARY_PATH not needed at link or run-time
gcc -o blah blah.o -lpcap -Wl,-{L,R}/opt/csw/lib

# This makes it possible to use relative paths; run `readelf -d binary_name`
# and you'll see '$ORIGIN/../lib/' in RPATH.  This plus `-zorigin` make it look
# relative to the binary for libraries at run-time
gcc -o blah blah.o -lsomelib -L/whatever/path/floats/your/boat -Wl,-R'$ORIGIN/../lib/' -Wl,-zorigin

.. where:

• paths given with -L are used at link-time
• paths given with -R are used at run-time