python - How to profile cython functions line-by-line

Question

I often struggle to find bottlenecks in my cython code. How can I profile cython functions line-by-line?

Answer 1

Robert Bradshaw helped me to get Robert Kern's line_profiler tool working for cdef functions and I thought I'd share the results on stackoverflow.

In short, set up a regular .pyx file and build script and add the following before your call to cythonize.

# Thanks to @tryptofame for proposing an updated snippet
from Cython.Compiler.Options import get_directive_defaults
directive_defaults = get_directive_defaults()

directive_defaults['linetrace'] = True
directive_defaults['binding'] = True

Furthermore, you need to define the C macro CYTHON_TRACE=1 by modifying your extensions setup such that

extensions = [
    Extension("test", ["test.pyx"], define_macros=[('CYTHON_TRACE', '1')])
]

A working example using the %%cython magic in the iPython notebook is here: http://nbviewer.ipython.org/gist/tillahoffmann/296501acea231cbdf5e7