Re: [HACKERS] [GENERAL] Floating point error

2013-03-06 Thread Florian Weimer
On 03/05/2013 07:23 PM, Tom Lane wrote: Maciek Sakrejda writes: Thank you: I think this is what I was missing, and what wasn't clear from the proposed doc patch. But then how can pg_dump assume that it's always safe to set extra_float_digits = 3? It's been proven (don't have a link handy, but

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Edson Richter
Em 05/03/2013 16:01, Tom Duffey escreveu: This conversation has moved beyond my ability to be useful but I want to remind everyone of my original issues in case it helps you improve the docs: 1) Data shown in psql did not match data retrieved by JDBC. I had to debug pretty deep into the JDBC c

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Tom Duffey
This conversation has moved beyond my ability to be useful but I want to remind everyone of my original issues in case it helps you improve the docs: 1) Data shown in psql did not match data retrieved by JDBC. I had to debug pretty deep into the JDBC code to confirm that a value I was staring at

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Tom Lane
Maciek Sakrejda writes: > On Tue, Mar 5, 2013 at 10:23 AM, Tom Lane wrote: >> Basically, the default behavior is tuned to the expectations of people >> who think that what they put in is what they should get back, ie we >> don't want the system doing this by default: >> >> regression=# set extra

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Maciek Sakrejda
On Tue, Mar 5, 2013 at 10:23 AM, Tom Lane wrote: >> Why the discrepancy between >> default behavior and what pg_dump gets? > > Basically, the default behavior is tuned to the expectations of people > who think that what they put in is what they should get back, ie we > don't want the system doing

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Tom Lane
Maciek Sakrejda writes: > Thank you: I think this is what I was missing, and what wasn't clear > from the proposed doc patch. But then how can pg_dump assume that it's > always safe to set extra_float_digits = 3? It's been proven (don't have a link handy, but the paper is at least a dozen years o

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread James Cloos
> "HL" == Heikki Linnakangas writes: HL> It would be nice to have a base-2 text format to represent floats. HL> It wouldn't be as human-friendly as base-10, but it could be used HL> when you don't want to lose precision. pg_dump in particular. hexidecimal notation for floats exists. The pri

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Maciek Sakrejda
On Tue, Mar 5, 2013 at 12:03 AM, Albe Laurenz wrote: > I don't think that it is about looking nice. > C doesn't promise you more than FLT_DIG or DBL_DIG digits of > precision, so PostgreSQL cannot either. > > If you allow more, that would mean that if you store the same > number on different platf

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Heikki Linnakangas
On 05.03.2013 15:59, Kevin Grittner wrote: Daniel Farina wrote: This kind of change may have many practical problems that may make it un-pragmatic to alter at this time (considering the workaround is to set the extra float digits), but I can't quite grasp the rationale for "well, the only prog

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Kevin Grittner
Daniel Farina wrote: > This kind of change may have many practical problems that may > make it un-pragmatic to alter at this time (considering the > workaround is to set the extra float digits), but I can't quite > grasp the rationale for "well, the only program that cares about > the most precis

Re: [HACKERS] [GENERAL] Floating point error

2013-03-05 Thread Albe Laurenz
Daniel Farina wrote: > On Mon, Mar 4, 2013 at 2:27 PM, Maciek Sakrejda wrote: >> On Sun, Mar 3, 2013 at 9:14 PM, Tom Lane wrote: >>> The real difficulty is that there may be more than one storable value >>> that corresponds to "1.23456" to six decimal digits. To be certain that >>> we can reprod

Re: [HACKERS] [GENERAL] Floating point error

2013-03-04 Thread Daniel Farina
On Mon, Mar 4, 2013 at 2:27 PM, Maciek Sakrejda wrote: > On Sun, Mar 3, 2013 at 9:14 PM, Tom Lane wrote: >> The real difficulty is that there may be more than one storable value >> that corresponds to "1.23456" to six decimal digits. To be certain that >> we can reproduce the stored value unique

Re: [HACKERS] [GENERAL] Floating point error

2013-03-04 Thread Maciek Sakrejda
On Sun, Mar 3, 2013 at 9:14 PM, Tom Lane wrote: > The real difficulty is that there may be more than one storable value > that corresponds to "1.23456" to six decimal digits. To be certain that > we can reproduce the stored value uniquely, we have to err in the other > direction, and print *more*

Re: [HACKERS] [GENERAL] Floating point error

2013-03-03 Thread Tom Lane
Maciek Sakrejda writes: > [ a bunch of questions that boil down to: ] > Isn't full fidelity possible assuming sensible rounding semantics and > enough characters of precision? The fundamental issue is that the underlying representation is binary and so its precision limit doesn't correspond to an

Re: [HACKERS] [GENERAL] Floating point error

2013-03-03 Thread Maciek Sakrejda
While having more docs around extra_float_digits is a great idea, I don't think the patch really clarifies much. (Disclaimer: I honestly have only a vague idea of the reasoning behind extra_float_digits existing in the first place, but perhaps that means I'm a good target audience for the doc patc

Re: [GENERAL] Floating point error

2013-03-02 Thread James Cloos
> "TD" == Tom Duffey writes: TD> We have a script thatb uses COPY to transfer a subset of rows TD> from a very large production table to a test table. The script TD> was not setting extra_float_digits so the values did not match TD> even though they appeared to match when running queries in p

Re: [GENERAL] Floating point error

2013-03-01 Thread Albe Laurenz
Tom Duffey wrote (on -general): > To bring closure to this thread, my whole problem was caused by not knowing > about the > extra_float_digits setting. We have a script that uses COPY to transfer a > subset of rows from a very > large production table to a test table. The script was not setting

Re: [GENERAL] Floating point error

2013-02-28 Thread Tom Duffey
Hi Everyone, To bring closure to this thread, my whole problem was caused by not knowing about the extra_float_digits setting. We have a script that uses COPY to transfer a subset of rows from a very large production table to a test table. The script was not setting extra_float_digits so the va

Re: [GENERAL] Floating point error

2013-02-28 Thread James Cloos
> "TD" == Tom Duffey writes: TD> Riddle me this. I have a database column of type "real" that gets TD> mapped to a Java field of type double via JDBC. ... TD> - Selecting values from both test and production DBs using psql TD> shows "10.3885" as the value TD> - The Java app on production

Re: [GENERAL] Floating point error

2013-02-25 Thread Kevin Grittner
Tom Duffey wrote: > CREATE TABLE test ( > id INTEGER PRIMARY KEY, > value REAL NOT NULL > ); > > INSERT INTO test (id, value) VALUES (1, 10.3884573), (2, 10.3885); > SELECT * FROM test; > > id |  value > +- >   1 | 10.3885 >   2 | 10.3885 > (2 rows) > > At this point you wou

Re: [GENERAL] Floating point error

2013-02-25 Thread Tom Duffey
That's exactly what I was looking for. We use COPY to transfer data from a 1 billion+ row table to a test database and were confused why the results looked the same but were obviously not. Sounds like we need to use the extra_float_digits setting to include all the available information when tr

Re: [GENERAL] Floating point error

2013-02-25 Thread Albe Laurenz
Tom Duffey wrote: > Here is a smaller test case that does not involve Java. I guess this probably > is just due to floating > point error when the initial value is inserted that is too large for the > field but it's still a > surprise. > > Create a test table, insert a couple values and view the

Re: [GENERAL] Floating point error

2013-02-25 Thread Tom Duffey
Here is a smaller test case that does not involve Java. I guess this probably is just due to floating point error when the initial value is inserted that is too large for the field but it's still a surprise. Create a test table, insert a couple values and view the results: CREATE TABLE test (

Re: [GENERAL] Floating point error

2013-02-24 Thread Adrian Klaver
On 02/24/2013 06:58 PM, Tom Duffey wrote: On Feb 24, 2013, at 8:44 PM, Adrian Klaver wrote: On 02/24/2013 06:13 PM, Tom Duffey wrote: Hi Everyone, Riddle me this. I have a database column of type "real" that gets mapped to a Java field of type double via JDBC. We have two databases, test a

Re: [GENERAL] Floating point error

2013-02-24 Thread Edson Richter
Em 24/02/2013 23:44, Adrian Klaver escreveu: On 02/24/2013 06:13 PM, Tom Duffey wrote: Hi Everyone, Riddle me this. I have a database column of type "real" that gets mapped to a Java field of type double via JDBC. We have two databases, test and production, and the test database is periodical

Re: [GENERAL] Floating point error

2013-02-24 Thread Tom Duffey
On Feb 24, 2013, at 8:44 PM, Adrian Klaver wrote: > On 02/24/2013 06:13 PM, Tom Duffey wrote: >> Hi Everyone, >> >> Riddle me this. I have a database column of type "real" that gets mapped to >> a Java field of type double via JDBC. We have two databases, test and >> production, and the test

Re: [GENERAL] Floating point error

2013-02-24 Thread Adrian Klaver
On 02/24/2013 06:13 PM, Tom Duffey wrote: Hi Everyone, Riddle me this. I have a database column of type "real" that gets mapped to a Java field of type double via JDBC. We have two databases, test and production, and the test database is periodically blown away and reloaded from a copy of prod

Re: [GENERAL] Floating point error

2013-02-24 Thread Edson Richter
Em 24/02/2013 23:26, John R Pierce escreveu: On 2/24/2013 6:20 PM, John R Pierce wrote: On 2/24/2013 6:13 PM, Tom Duffey wrote: - The Java app on production shows "10.3884573" while the test app shows "10.3885" 'real' is single precision, which is only about 6 digits of decimal accuracy. i

Re: [GENERAL] Floating point error

2013-02-24 Thread John R Pierce
On 2/24/2013 6:20 PM, John R Pierce wrote: On 2/24/2013 6:13 PM, Tom Duffey wrote: - The Java app on production shows "10.3884573" while the test app shows "10.3885" 'real' is single precision, which is only about 6 digits of decimal accuracy. if your java variables were double precision, y

Re: [GENERAL] Floating point error

2013-02-24 Thread John R Pierce
On 2/24/2013 6:13 PM, Tom Duffey wrote: - The Java app on production shows "10.3884573" while the test app shows "10.3885" 'real' is single precision, which is only about 6 digits of decimal accuracy. if your java variables were double precision, you probably should have used double in pos

[GENERAL] Floating point error

2013-02-24 Thread Tom Duffey
Hi Everyone, Riddle me this. I have a database column of type "real" that gets mapped to a Java field of type double via JDBC. We have two databases, test and production, and the test database is periodically blown away and reloaded from a copy of production. We recently noticed that some value