The problem at hand is that the variable singleton
exists twice: once in the extension setter
and once in the extension getter
(also functions get_singleton
and set_singleton
exist twice, i.e. having two different addresses each), this is more or less a violation of one definition rule (ODR) even if this rule only exists in C++. Violation of ODR is not the end of the world, but in most cases the behavior becomes not portable because different linkers/compilers/OSes handle this situation differently.
For example, for shared libraries on Linux we have the symbol-interposition. However, Python uses ldopen
without RTLD_GLOBAL
(means implicitly with RTLD_LOCAL
) for loading of C-extensions, thus preventing symbol-interposition. We could enforce the usage of RTLD_GLOBAL
in Python:
import sys; import ctypes;
sys.setdlopenflags(sys.getdlopenflags() | ctypes.RTLD_GLOBAL)
prior to importing getter
and setter
and restore the singleton-property again. This however would not work on Windows, because dlls don't support symbol-interposition.
The portable way to ensure the "singleton-property" is to avoid a violation of ODR and in order to achieve that one should make the static library/bunch of files dynamic. This dynamic library will be loaded only once by the process, thus ensuring that we have only one singleton
.
Depending on the scenario, there are some options how this dll can be used:
- the extensions are used only locally and not distributed, using shared objects (see this SO-post) or dll (see this SO-post).
- extensions are only distributed on some platforms, then it is possible to prebuild shared objects/dlls and distribute them like a third-party library, see for example this SO-post
- it is possible to override setuptools'
build_clib
command, so it will build a shared object/dll instead of a static library, which will be used when extensions are linked and copied to the installation. However, adding more extension, which would use this very dll is quite cumbersome (even if not impossible).
- it is possible to write a cython-wrapper of the dll, which has the advantage of using the Python's machinery for loading and posponing the resultion of the symbols until the run-time, rather than linkers/loaders of the underlying OSes, which makes it for example easier to create futher extensions depending on the dynamic library later on.
I think the last approach should be used by default. Here is a possible implementation:
- Create a wrapper of the static library and expose its functionality via a
pxd
-file:
# lib_wrapper.pxd
cdef int get_singleton()
cdef void set_singleton(int new_value)
#lib_wrapper.pyx
cdef extern from "lib.h":
int c_get_singleton "get_singleton" ()
void c_set_singleton "set_singleton" (int new_val)
cdef int get_singleton():
return c_get_singleton()
cdef void set_singleton(int new_val):
c_set_singleton(new_val)
An important part: the wrapper introduces a level of indirection (thus inducing a lot of boiler-plate code writing which should be automatized), thus when using it in further modules neither header-files nor c-files/libraries are needed.
- Adjust other modules, they need only to
cimport
the wrapper:
# getter.pyx:
#cython: language_level=3
cimport lib_wrapper
def get():
return lib_wrapper.get_singleton()
# setter.pyx:
#cython: language_level=3
cimport lib_wrapper
def set(new_val):
lib_wrapper.set_singleton(new_val)
- Setup no longer needs
build_clib
-step:
from setuptools import setup, find_packages, Extension
setup(
name='singleton_example',
ext_modules=[Extension('lib_wrapper', sources=['lib_wrapper.pyx', 'lib.c']),
Extension('getter', sources=['getter.pyx']),
Extension('setter', sources=['setter.pyx']),],
)
After building via python setup.py build_ext --inplace
(in the source distribution, i.e. python setup.py build sdist
the h-file will be missing, but there are many different solutions for this problem possible) the example will set
/get
the same singleton (because there is only one).