python 2.7 - Matplotlib OverflowError: Allocated too many blocks

Question

I am plotting three sets of roughly 20 time series. Each time series has 1.8 million points and I am plotting them separately. In order to reduce execution time (because I found out that the figure() function is quite time consuming), I open one figure only, plot and then clean it for the next time series. Something like:

import matplotlib.pyplot as plt

fig = plt.figure() # open a figure for plotting
for v in range(nfigs):
    # here I build the x and y vectors from the raw data
    # and also the figname string
    plt.plot(x, y, linewidth=0.3)
    plt.title('a title')
    plt.xlabel('x-label')
    plt.ylabel('y-label')
    plt.grid()
    plt.draw()
    plt.savefig(figname)
    plt.clf()
plt.close() # close the figure after the job is done

All the plots are saved ok except for one in particular in all the data sets which gives me the following error (edited the file paths for privacy):

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "~AppDataLocalContinuumAnacondalibsite-packagesspyderlibwidgetsexternalshellsitecustomize.py", line 540, in runfile
    execfile(filename, namespace)
  File "batch.py", line 142, in <module>
    main(fn)
  File "batch.py", line 94, in main
    plt.savefig(figname+'_'+nn+'.png')
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibpyplot.py", line 561, in savefig
    return fig.savefig(*args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibfigure.py", line 1421, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibackendsackend_qt4agg.py", line 167, in print_figure
    FigureCanvasAgg.print_figure(self, *args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibackend_bases.py", line 2220, in print_figure
    **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibackendsackend_agg.py", line 505, in print_png
    FigureCanvasAgg.draw(self)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibackendsackend_agg.py", line 451, in draw
    self.figure.draw(self.renderer)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibartist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibfigure.py", line 1034, in draw
    func(*args)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibartist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibaxes.py", line 2086, in draw
    a.draw(renderer)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibartist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotliblines.py", line 562, in draw
    drawFunc(renderer, gc, tpath, affine.frozen())
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotliblines.py", line 938, in _draw_lines
    self._lineFunc(renderer, gc, path, trans)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotliblines.py", line 978, in _draw_solid
    renderer.draw_path(gc, path, trans)
  File "~AppDataLocalContinuumAnacondalibsite-packagesmatplotlibackendsackend_agg.py", line 145, in draw_path
    self._renderer.draw_path(gc, path, transform, rgbFace)
OverflowError: Allocated too many blocks

The time series that triggers this error has absolutely nothing special about it, I have checked it with a simple show() instead of savefig(). It is neither the first one nor the last one (in roughly 60 time series) but it is enough to break my script. Help on this is much appreciated!

Answer 1

You can try to change the value of agg.path.chunksize (10000-100000) in the matplotlibrc file or during run time in matplotlib.rcParams. It causes the renderer to plot paths in chunks rather than the whole path at once; but according to the docs it might cause artifacts. See the sample matplotlibrc file at http://matplotlib.org/1.3.1/users/customizing.html for some more information.