On Wed, Sep 1, 2010 at 2:26 PM, Michael Gilbert <michael.s.gilb...@gmail.com > wrote:
> Hi, > > I've been using numpy's float96 class lately, and I've run into some > strange precision errors. See example below: > > >>> import numpy > >>> numpy.version.version > '1.5.0' > >>> sys.version > '3.1.2 (release31-maint, Jul 8 2010, 01:16:48) \n[GCC 4.4.4]' > >>> x = numpy.array( [0.01] , numpy.float32 ) > >>> y = numpy.array( [0.0001] , numpy.float32 ) > >>> x[0]*x[0] - y[0] > 0.0 > >>> x = numpy.array( [0.01] , numpy.float64 ) > >>> y = numpy.array( [0.0001] , numpy.float64 ) > >>> x[0]*x[0] - y[0] > 0.0 > >>> x = numpy.array( [0.01] , numpy.float96 ) > >>> y = numpy.array( [0.0001] , numpy.float96 ) > >>> x[0]*x[0] - y[0] > -6.286572655403010329e-22 > > I would expect the float96 calculation to also produce 0.0 exactly as > found in the float32 and float64 examples. Why isn't this the case? > > None of the numbers is exactly represented in ieee floating format, so what you are seeing is rounding error. Note that the first two zeros are only accurate to about 7 and 16 digits respectively, whereas fot float 96 is accurate to about 19 digits. Slightly off-topic: why was the float128 class dropped? > > It wasn't, but you won't see it on a 32 bit system because of how the gcc compiler treats long doubles for alignment reasons. For common intel hardware/os, float96, and float128 are the same precision, just stored differently. In general the long precision formats are not portable, so watch out. Chuck
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion