On Tue, Nov 30, 2010 at 12:29 AM, Shane Harrelson wrote:
>
>
I've written two articles about this topic that summarize the problem and
how it may be fixed:
-
http://www.exploringbinary.com/fifteen-digits-dont-round-trip-through-sqlite-reals/
This covers the original mailing list discu
On Tue, Nov 30, 2010 at 12:29 AM, Shane Harrelson wrote:
>
It turns out the same problem exists on Linux (I used sqlite3-3.7.3.bin);
for example:
sqlite> create table t1(d float);
sqlite> insert into t1 values(9.87e+31);
sqlite> select * from t1;
9.870001e+31
So it seems that extende
In Visual C, a "long double" and a "double" are the same, and only
offer 53 bits of precision.
On GCC, a "long double" has 80 bits of precision.
Unfortunately, I don't think there's a way to have Visual C use more precision.
Because of this, round off error will always differ between the two
compi
On Mon, Nov 29, 2010 at 3:04 PM, Rick Regan wrote:
> So the question remains -- why does the Windows build get it wrong?
>
I think I figured it out. It is due to the use of extended precision. In
sqlite3.c, if you change "#define LONGDOUBLE_TYPE long double" to "#define
LONGDOUBLE_TYPE double"
On Sun, Nov 28, 2010 at 8:55 AM, Black, Michael (IS) wrote:
>
> I find it interesting though that the math is different between the two
> compilers...I wonder which is actually "correct" ??
>
>
>
I ran a GMP version of the section of code we've highlighted, converting all
the double variables to
On Mon, Nov 29, 2010 at 10:08 AM, Doug Currie wrote:
>
> The Grisu2 strings do "round trip" correctly, which I think is where this
> thread started.
>
>
Grisu addresses the "floating-point to decimal to floating-point"
round-trip. The bug I found (it will become a bug report, right?) concerns
the
On Nov 29, 2010, at 9:37 AM, Rick Regan wrote:
> "For IEEE 754 double-precision numbers and 64-bit integers roughly
> 99.4% of all numbers can be processed efficiently. The remaining 0.6% are
> rejected and need to be printed by a slower complete algorithm."
>
> Hmmm. What's involved in the "sl
On Sun, Nov 28, 2010 at 8:09 PM, Doug Currie wrote:
> The paper compares the performance of sprintf from glibc 2.11 and Grisu. I
> don't know if glibc sprintf is based on Gay's code; at one point I thought
> it was, but I cannot find an authoritative reference.
>
> In any case, the Grisus benchma
On Nov 28, 2010, at 6:19 PM, Rick Regan wrote:
> On Sun, Nov 28, 2010 at 5:52 PM, Doug Currie wrote:
>
>>
>> There is a new publication on this subject that may be of interest to those
>> looking at providing solutions:
>>
>> http://portal.acm.org/citation.cfm?id=1806623
>>
>> It (Grisu2) wo
On Sun, Nov 28, 2010 at 5:52 PM, Doug Currie wrote:
>
> There is a new publication on this subject that may be of interest to those
> looking at providing solutions:
>
> http://portal.acm.org/citation.cfm?id=1806623
>
> It (Grisu2) works without bignums if you are willing to settle for the
> shor
On Nov 28, 2010, at 5:37 PM, Rick Regan wrote:
> On Sun, Nov 28, 2010 at 4:01 PM, Doug Currie wrote:
>
>> On Nov 28, 2010, at 11:18 AM, Rick Regan wrote:
>>
>>> Michael,
>>> Thanks for the very thorough analysis.
>>
>> This is a difficult problem; fortunately it was solved 20 years ago...
>>
On Sun, Nov 28, 2010 at 4:01 PM, Doug Currie wrote:
> On Nov 28, 2010, at 11:18 AM, Rick Regan wrote:
>
> > Michael,
> > Thanks for the very thorough analysis.
>
> This is a difficult problem; fortunately it was solved 20 years ago...
>
> Well, it's not solved on Windows. :)
On Nov 28, 2010, at 11:18 AM, Rick Regan wrote:
> Michael,
> Thanks for the very thorough analysis.
This is a difficult problem; fortunately it was solved 20 years ago...
http://www.mail-archive.com/sqlite-users@sqlite.org/msg09529.html
e
___
sqlite-
BTW, I get the CORRECT result when running the precompiled command line on
Linux:
SQLite version 3.7.3
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> create table t1(d float);
sqlite> insert into t1 values(8.948471e15);
sqlite> select * from t1;
8.948471e+15
Michael,
Thanks for the very thorough analysis.
Yes, sorry, I forgot to state I was running on Windows. The results I got
came from running the command window (the precompiled Windows binary
sqlite-3_7_3.zip). I also got the same results from compiling the
amalgamation (sqlite-amalgamation-3_7_3.
ion Systems
From: sqlite-users-boun...@sqlite.org on behalf of Rick Regan
Sent: Sat 11/27/2010 11:09 AM
To: sqlite-users@sqlite.org
Subject: EXTERNAL:[sqlite] Some floats of 15 digits or less do not round-trip
I expected floats of 15 significant decimal di
I expected floats of 15 significant decimal digits or less to round-trip:
sqlite> create table t1(d float);
sqlite> insert into t1 values(8.948471e15);
sqlite> select * from t1;
8.9484710001e+15
The double value stored in the database is correct (I traced the code and
printed the value of 're
17 matches
Mail list logo
