The short answer is "dont' confuse numpy.finfo
with numpy.spacing
".
finfo
operates on the dtype
, while spacing
operates on the value.
Background Information
First, though, some general explanation:
The key part to understand is that floating point numbers are similar to scientific notation. Just like you'd write 0.000001 as 1.0 x 10^-6
, floats are similar to c x 2^q
. In other words, they have two separate parts - a coefficient (c
, a.k.a. "significand") and an exponent (q
). These two values are stored as integers.
Therefore, how closely a value can be represented (let's think of this as the degree of discretization) is a function of both parts, and depends on the magnitude of the value.
However, the "precision" (as referred to by np.finfo
) is essentially the number of significant digits if the number were written in base-10 scientific notation. The "resolution" is the resolution of the coefficient (part in front) if the value were written in the same base-10 scientific notation (i.e. 10^-precision
). In other words, both are only a function of the coefficient.
Numpy-specific
For numpy.finfo
, "precision" and "resolution" are simply the inverse of each other. Neither one tells you how closely a particular number is being represented. They're purely a function of the dtype
.
Instead, if you're worried about the absolute degree of discretization, use numpy.spacing(your_float)
. This will return the difference in the next largest value in that particular format (e.g. it's different for a float32
than a float64
).
Examples
As an example:
In [1]: import numpy as np
In [2]: np.spacing(10.1)
Out[2]: 1.7763568394002505e-15
In [3]: np.spacing(10000000000.1)
Out[3]: 1.9073486328125e-06
In [4]: np.spacing(1000000000000.1)
Out[4]: 0.0001220703125
In [5]: np.spacing(100000000000000.1)
Out[5]: 0.015625
In [6]: np.spacing(10000000000000000.1)
Out[6]: 2.0
But the precision and resolution don't change:
In [7]: np.finfo(10.1).precision
Out[7]: 15
In [8]: np.finfo(10000000000000000.1).precision
Out[8]: 15
In [9]: np.finfo(10.1).resolution
Out[9]: 1.0000000000000001e-15
In [10]: np.finfo(10000000000000000000.1).resolution
Out[10]: 1.0000000000000001e-15
Also note that all of these depend on the data type that you're using:
In [11]: np.spacing(np.float32(10.1))
Out[11]: 9.5367432e-07
In [12]: np.spacing(np.float32(10000000000000.1))
Out[12]: 1048576.0
In [13]: np.finfo(np.float32).precision
Out[13]: 6
In [14]: np.finfo(np.float32).resolution
Out[14]: 1e-06
In [15]: np.spacing(np.float128(10.1))
Out[15]: 8.6736173798840354721e-19
In [16]: np.spacing(np.float128(10000000000000.1))
Out[16]: 9.5367431640625e-07
In [17]: np.finfo(np.float128).precision
Out[17]: 18
In [18]: np.finfo(np.float128).resolution
Out[18]: 1.0000000000000000007e-18
Specific Questions
Now on to your specific questions:
But practically, does it mean that I should expect results to be erroneous if I preform operations using numbers less than the resolution?
No, because the precision/resolution (in numpy.finfo
terms) is only a function of the coefficient, and doesn't take into account the exponent. Very small and very large numbers have the same "precision", but that's not an absolute "error".
As a rule of thumb, when using the "resolution" or "precision" terms from finfo
, think of scientific notation. If we're operating on small numbers with similar magnitudes, we don't need to worry about much.
Let's take the decimal math case with 6 significant digits (somewhat similar to a float32
):
1.20000 x 10^-19 + 2.50000 x 10^-20 => 1.45000 x 10^19
However, if we operate on numbers with wildly different magnitudes but limited precision (again, 6 significant digits):
1.20000 x 10^6 + 2.50000 x 10^-5 => 1.20000
We'll start to see the effects quite clearly.
How can I quantify the error, for say addition, of two floating point numbers given their precision?
Use np.spacing(result)
.
If the resolution is as "large" as 1e-15, why would the smallest allowable number be on the order of 1e-308?
Again, the "resolution" in this case doesn't take into account the exponent, just the part in front.
Hopefully that helps clarify things somewhat. All of this is a bit confusing, and everyone gets bitten by it at some point. It's good to try to build up a bit of intuition about it and to know what functions to call to find out exactly in your platform-of-choice!