math - Linear regression of arrays containing NANs in Python/Numpy

Question

I have two arrays, say varx and vary. Both contain NAN values at various positions. However, I would like to do a linear regression on both to show how much the two arrays correlate. This was very helpful so far: http://glowingpython.blogspot.de/2012/03/linear-regression-with-numpy.html

However, using this:

slope, intercept, r_value, p_value, std_err = stats.linregress(varx, vary)

results in nans for every output variable. What is the most convenient way to take only valid values from both arrays as input to the linear regression? I heard about masking arrays, but am not sure how it works exactly.

Answer 1

You can remove NaNs using a mask:

mask = ~np.isnan(varx) & ~np.isnan(vary)
slope, intercept, r_value, p_value, std_err = stats.linregress(varx[mask], vary[mask])