On Tue, 16 Dec 2014, Hartmut Holzgraefe wrote:
On 16.12.2014 15:16, xiangdongzou wrote:
Can anyone tell me why 531808.11 has been changed to 531808.12 ?
typical decimal->binary->decimal conversion/rounding error.
never used DECIMAL nor intend to, but the issue is typical of precision
issues among float (32-bit) and double (64-bit) in ANY programming
language. Google for "IEEE floating point"
Some (most) users are unaware that a 32-bit real (REAL*4 for oldtimer
Fortran users like myself) have about 7 digits of precision, while 64-bit
(doubles, double precision, REAL*8) get to about 16.
So if a quantity needs high precision (typically this occurs for angular
quantities where arcseconds are important), use double.
--
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
------------------------------------------------------------------------
Do not like Firefox >=29 ? Get Pale Moon ! http://www.palemoon.org
--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql
To unsubscribe: http://lists.mysql.com/mysql
