Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Robert Haas
On Wed, Aug 4, 2010 at 12:55 PM, Tom Lane wrote: > Robert Haas writes: >> *scratches head* > >> One of those tests uses < and the other uses <= > >> Which one is wrong? > > Well, neither, since NUMERIC(0,0) is disallowed.  However, it's probably > not the place of these functions to assume that,

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Tom Lane
Robert Haas writes: > *scratches head* > One of those tests uses < and the other uses <= > Which one is wrong? Well, neither, since NUMERIC(0,0) is disallowed. However, it's probably not the place of these functions to assume that, so I'd suggest treating equality as valid.

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Robert Haas
On Wed, Aug 4, 2010 at 11:05 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas wrote: >>> The smallest value for precision which requires 2 numeric_digits is >>> always 2; and the required number of numeric_digits increases by 1 >>> each time the number of

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas wrote: >> The smallest value for precision which requires 2 numeric_digits is >> always 2; and the required number of numeric_digits increases by 1 >> each time the number of base-10 digits increases by DEC_DIGITS. > And here is

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-03 Thread Robert Haas
On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas wrote: > On Fri, Jul 30, 2010 at 2:08 PM, Tom Lane wrote: >> Robert Haas writes: >>>  Maybe something like this, >>> obviously with a suitable comment which I haven't written yet: >> >>>     numeric_digits = (precision + 6) / 4; >>>     return (n

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Robert Haas
On Tue, Aug 3, 2010 at 6:03 PM, Tom Lane wrote: > Robert Haas writes: >> Here's a second version of the main patch, in which I have attempted >> to respond to Tom's concerns/suggestions. > > This version looks fine to me. Excellent. Committed. -- Robert Haas EnterpriseDB: http://www.enterpris

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Tom Lane
Robert Haas writes: > Here's a second version of the main patch, in which I have attempted > to respond to Tom's concerns/suggestions. This version looks fine to me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Brendan Jurd
On 31 July 2010 07:58, Robert Haas wrote: > Here's a second version of the main patch, in which I have attempted > to respond to Tom's concerns/suggestions. > > (There is still a small, side issue with numeric_maximum_size() which > I plan to fix, but this patch is the good stuff.) > Applies fine

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 2:08 PM, Tom Lane wrote: > Robert Haas writes: >>  Maybe something like this, >> obviously with a suitable comment which I haven't written yet: > >>     numeric_digits = (precision + 6) / 4; >>     return (numeric_digits * sizeof(int16)) + NUMERIC_HDRSZ; > > This is O

[HACKERS] reducing NUMERIC size for 9.1, take two

2010-07-30 Thread Robert Haas
Here's a second version of the main patch, in which I have attempted to respond to Tom's concerns/suggestions. (There is still a small, side issue with numeric_maximum_size() which I plan to fix, but this patch is the good stuff.) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Ente

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 9:16 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane wrote: >>> Perhaps, but I think you're confused on at least one point. >>> numeric(2,1) has to be able to hold 2 decimal digits, not 2 >>> NumericDigits (which'd actually be 8 deci

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
[ forgot to answer this part ] Robert Haas writes: > Another question here is whether we should just fix this in CVS HEAD, > or whether there's any reason to back-patch it. Agreed, fixing it in HEAD seems sufficient. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
Robert Haas writes: > Maybe something like this, > obviously with a suitable comment which I haven't written yet: > numeric_digits = (precision + 6) / 4; > return (numeric_digits * sizeof(int16)) + NUMERIC_HDRSZ; This is OK for the base-10K case, but there's still code in there for

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
Robert Haas writes: > On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane wrote: >> Perhaps, but I think you're confused on at least one point. >> numeric(2,1) has to be able to hold 2 decimal digits, not 2 >> NumericDigits (which'd actually be 8 decimal digits given >> the current code). > I get that. T

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane wrote: > Robert Haas writes: >> But, looking at it a bit more carefully, isn't the maximum-size logic >> for numeric rather bogus? > > Perhaps, but I think you're confused on at least one point. > numeric(2,1) has to be able to hold 2 decimal digits, not

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas writes: > But, looking at it a bit more carefully, isn't the maximum-size logic > for numeric rather bogus? Perhaps, but I think you're confused on at least one point. numeric(2,1) has to be able to hold 2 decimal digits, not 2 NumericDigits (which'd actually be 8 decimal digits given

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 5:35 PM, Tom Lane wrote: > Robert Haas writes: >> Did you look at the >> patch to move the numeric stuff out of the .h file that I attached a >> few emails back?  If that looks OK, I can commit it and then redo the >> rest of this along the lines we've discussed. > > A cou

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas writes: > Did you look at the > patch to move the numeric stuff out of the .h file that I attached a > few emails back? If that looks OK, I can commit it and then redo the > rest of this along the lines we've discussed. A couple of thoughts: * It'd be good to get NUMERIC_HDRSZ out o

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 5:03 PM, Tom Lane wrote: > Robert Haas writes: >> OK.  I think you're misinterpreting the point of that comment, which >> may mean that it needs some clarification.  By "the two byte header >> format is also used", I think I really meant "the header (and in fact >> the ent

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas writes: > OK. I think you're misinterpreting the point of that comment, which > may mean that it needs some clarification. By "the two byte header > format is also used", I think I really meant "the header (and in fact > the entire value) is just 2 bytes". Really, the low order bits

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 4:37 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane wrote: >>> On-disk is what I'm thinking about.  Right now, a NaN's first word is >>> all dscale except the sign bits.  You're proposing to change that >>> but I think it's unnecess

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas writes: > On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane wrote: >> On-disk is what I'm thinking about.  Right now, a NaN's first word is >> all dscale except the sign bits.  You're proposing to change that >> but I think it's unnecessary to do so. > *Where* am I proposing this? Um, your

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane wrote: > Yeah, you would need an additional layer of struct to represent the > numeric with a length word in front of it.  I think this is not > necessarily bad because it would perhaps open the door to working > directly with short-varlena-header values,

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas writes: > On Wed, Jul 28, 2010 at 3:00 PM, Tom Lane wrote: >> No, you can do something like this: >> >> typedef union numeric >> { >>uint16 word1; >>numeric_short short; >>numeric_longlong; >> } numeric; > That doesn't quite work because there's also a

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Wed, Jul 28, 2010 at 3:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane wrote: >>> I don't like the way you did that either (specifically, not the kluge >>> in NUMERIC_DIGITS()).  It would probably work better if you declared >>> two different structs,

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-28 Thread Tom Lane
[ gradually catching up on email ] Robert Haas writes: > On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane wrote: >> I don't like the way you did that either (specifically, not the kluge >> in NUMERIC_DIGITS()).  It would probably work better if you declared >> two different structs, or a union of same,

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-20 Thread Robert Haas
On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane wrote: > Robert Haas writes: >> I'm not entirely happy with the way I handled the variable-length >> struct, although I don't think it's horrible, either. I'm willing to >> rework it if someone has a better idea. > > I don't like the way you did that eith

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Tom Lane
Robert Haas writes: > I'm not entirely happy with the way I handled the variable-length > struct, although I don't think it's horrible, either. I'm willing to > rework it if someone has a better idea. I don't like the way you did that either (specifically, not the kluge in NUMERIC_DIGITS()). It

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Hitoshi Harada
2010/7/16 Brendan Jurd : > On 16 July 2010 03:47, Robert Haas wrote: >> You might also look at testing with pg_column_size(). >> > > pg_column_size() did return the results I was expecting. > pg_column_size(0::numeric) is 8 bytes on 8.4 and it's 6 bytes on HEAD > with your patch. > > However, even

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread David E. Wheeler
On Jul 16, 2010, at 9:04 AM, Bruce Momjian wrote: >> What are the implications for pg_upgrade? Will a database with values >> created before the patch continue to work after the patch has been >> applied (as happened with the new hstore in 9.0), or will pg_upgrade >> need to be taught how to upgra

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Bruce Momjian
David E. Wheeler wrote: > On Jul 16, 2010, at 6:17 AM, Thom Brown wrote: > > > Joy! :) Nice patch Robert. > > Indeed. > > What are the implications for pg_upgrade? Will a database with values > created before the patch continue to work after the patch has been > applied (as happened with the ne

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread David E. Wheeler
On Jul 16, 2010, at 6:17 AM, Thom Brown wrote: > Joy! :) Nice patch Robert. Indeed. What are the implications for pg_upgrade? Will a database with values created before the patch continue to work after the patch has been applied (as happened with the new hstore in 9.0), or will pg_upgrade nee

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Thom Brown
On 16 July 2010 14:14, Brendan Jurd wrote: > On 16 July 2010 22:51, Richard Huxton wrote: >> On 16/07/10 13:44, Brendan Jurd wrote:> >>> At this scale we should be seeing around 2 million bytes saved, but >>> instead the tables are identical.  Is there some kind of disconnect in >>> how the new s

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Brendan Jurd
On 16 July 2010 22:51, Richard Huxton wrote: > On 16/07/10 13:44, Brendan Jurd wrote:> >> At this scale we should be seeing around 2 million bytes saved, but >> instead the tables are identical.  Is there some kind of disconnect in >> how the new short numeric is making it to the disk, or perhaps

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Richard Huxton
On 16/07/10 13:44, Brendan Jurd wrote: pg_column_size() did return the results I was expecting. pg_column_size(0::numeric) is 8 bytes on 8.4 and it's 6 bytes on HEAD with your patch. At this scale we should be seeing around 2 million bytes saved, but instead the tables are identical. Is ther

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Brendan Jurd
On 16 July 2010 03:47, Robert Haas wrote: > On Jul 15, 2010, at 11:58 AM, Brendan Jurd wrote: >> I dropped one thousand numerics with value zero into a table and >> checked the on-disk size of the relation with your patch and on a >> stock 8.4 instance.  In both cases the result was exactly the s

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-15 Thread Robert Haas
On Jul 15, 2010, at 11:58 AM, Brendan Jurd wrote: > On 10 July 2010 00:58, Robert Haas wrote: >> EnterpriseDB asked me to develop the attached patch to reduce the >> on-disk size of numeric and to submit it for inclusion in PG 9.1. >> After searching the archives, I found a possible design for th

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-15 Thread Brendan Jurd
On 10 July 2010 00:58, Robert Haas wrote: > EnterpriseDB asked me to develop the attached patch to reduce the > on-disk size of numeric and to submit it for inclusion in PG 9.1. > After searching the archives, I found a possible design for this by > Tom Lane based on an earlier proposal by Simon R

[HACKERS] reducing NUMERIC size for 9.1

2010-07-09 Thread Robert Haas
EnterpriseDB asked me to develop the attached patch to reduce the on-disk size of numeric and to submit it for inclusion in PG 9.1. After searching the archives, I found a possible design for this by Tom Lane based on an earlier proposal by Simon Riggs. http://archives.postgresql.org/pgsql-hackers