Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom ... btw, has anyone noticed that this patch broke hamerkop and Tom bowerbird? Or at least, it's hard to see what other recent commit Tom would explain the failures they're showing. Now that Robert committed the fix for 64bit Datum w/o

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: For those following along at home, the failures are on these queries: SELECT 1.1 AS two UNION SELECT 2.2; SELECT 1.1 AS two UNION SELECT 2; SELECT 1 AS two UNION SELECT 2.2; SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2; In each

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: I see nothing in the win32 stuff that tries to define USE_FLOAT8_BYVAL on 64-bit windows, is this just an oversight or does it not actually work there? or is it for on-disk compatibility with 32-bit windows? Tom That flag doesn't affect

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes: Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom ... btw, has anyone noticed that this patch broke hamerkop and Tom bowerbird? Or at least, it's hard to see what other recent commit Tom would explain the failures they're showing. Now that

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Robert Haas
On Sat, Apr 4, 2015 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: I see nothing in the win32 stuff that tries to define USE_FLOAT8_BYVAL on 64-bit windows, is this just an oversight or does it not actually work there? or is it for on-disk compatibility with 32-bit windows? That flag doesn't

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sat, Apr 4, 2015 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: Having said that, I'm fine with leaving this as-is, if only because it means we're exercising the --disable-float8-byval code paths in the buildfarm ;-) But... are we? I mean, I

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Andrew Dunstan
On 04/04/2015 10:27 AM, Tom Lane wrote: Andrew Gierth and...@tao11.riddles.org.uk writes: Tom == Tom Lane t...@sss.pgh.pa.us writes: Tom ... btw, has anyone noticed that this patch broke hamerkop and Tom bowerbird? Or at least, it's hard to see what other recent commit Tom would explain

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Peter Geoghegan
On Sat, Apr 4, 2015 at 4:37 PM, Andrew Dunstan and...@dunslane.net wrote: This seems quite wrong. If we want those paths tested we should ensure that buildfarm members are set up with that explicit setting. I think not making this the default for 64 bit MSVC builds was simply an error of

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Andrew Dunstan
On 04/04/2015 04:27 PM, Robert Haas wrote: On Sat, Apr 4, 2015 at 10:27 AM, Tom Lane t...@sss.pgh.pa.us wrote: I see nothing in the win32 stuff that tries to define USE_FLOAT8_BYVAL on 64-bit windows, is this just an oversight or does it not actually work there? or is it for on-disk

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-04 Thread Robert Haas
On Apr 4, 2015, at 4:37 PM, Andrew Dunstan and...@dunslane.net wrote: The attached patch should set float8byval as the default on 64 bit MSVC builds and error out if it is explicitly set on 32 bit platforms. +1 for changing this. ...Robert -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Robert Haas
On Thu, Apr 2, 2015 at 10:41 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Robert - Changed some definitions to depend on SIZEOF_DATUM rather Robert than USE_FLOAT8_BYVAL. Hopefully I didn't muff this; please Robert check it. No, that's wrong; it must use USE_FLOAT8_BYVAL since

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Andrew Gierth
Robert == Robert Haas robertmh...@gmail.com writes: No, that's wrong; it must use USE_FLOAT8_BYVAL since that's what the definitions of Int64GetDatum / DatumGetInt64 are conditional on. As you have it now, it'll break if you build with --disable-float8-byval on a 64bit platform. Robert

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Andrew Gierth
Robert == Robert Haas robertmh...@gmail.com writes: I don't consider it appropriate to override ./configure in this way. Robert I don't see how that's overriding configure. Commit Robert 8472bf7a73487b0535c95e299773b882f7523463, which introduced Robert --disable-float8-byval in 2008,

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Robert Haas
On Fri, Apr 3, 2015 at 1:39 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: It already does; it changes how int64 values are expected to be stored in Datum variables. _Everything_ that currently stores an int64 value in a Datum is affected. But this isn't a value of the SQL type int64.

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Tom Lane
... btw, has anyone noticed that this patch broke hamerkop and bowerbird? Or at least, it's hard to see what other recent commit would explain the failures they're showing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Tom Lane
I wrote: FWIW, I think it's sensible to define NumericAbbrevGetDatum and the converse, but I'd suggest you just do it like #define NumericAbbrevGetDatum(X) Int64GetDatum(X) or #define NumericAbbrevGetDatum(X) Int32GetDatum(X) Oh, scratch that, I'd failed to look at the upthread messages.

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Andrew Gierth
Robert == Robert Haas robertmh...@gmail.com writes: It already does; it changes how int64 values are expected to be stored in Datum variables. _Everything_ that currently stores an int64 value in a Datum is affected. Robert But this isn't a value of the SQL type int64. It's just a

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Apr 3, 2015 at 1:39 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: If you're determined to go this route - over my protest - then you need to do something like define a NumericAbbrevGetDatum(x) macro and use it in place of the

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: For those following along at home, the failures are on these queries: SELECT 1.1 AS two UNION SELECT 2.2; SELECT 1.1 AS two UNION SELECT 2; SELECT 1 AS two UNION SELECT 2.2; SELECT 1.1 AS three UNION SELECT 2 UNION ALL SELECT 2; In each case, the

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Robert Haas
On Fri, Apr 3, 2015 at 5:45 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... btw, has anyone noticed that this patch broke hamerkop and bowerbird? Or at least, it's hard to see what other recent commit would explain the failures they're showing. I noticed the failure on bowerbird today and I agree

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Robert Haas
On Fri, Apr 3, 2015 at 3:46 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: (This does rather suggest to me that some better regression tests for sorting would be a good idea, possibly even including on-disk sorts.) Yeah. I've been unpleasantly surprised by how easy it is to pass the

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-03 Thread Robert Haas
On Fri, Apr 3, 2015 at 9:06 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Robert == Robert Haas robertmh...@gmail.com writes: No, that's wrong; it must use USE_FLOAT8_BYVAL since that's what the definitions of Int64GetDatum / DatumGetInt64 are conditional on. As you have it now,

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Petr Jelinek
On 02/04/15 21:21, Robert Haas wrote: On Thu, Apr 2, 2015 at 2:07 PM, Robert Haas robertmh...@gmail.com wrote: I think this is really nice work, so I have committed this version. I made a few fairly minor changes, hopefully without breaking anything in the process: - I adjusted things for

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Robert Haas
On Thu, Apr 2, 2015 at 4:24 PM, Petr Jelinek p...@2ndquadrant.com wrote: Thanks. I actually pushed a fix for that about 25 minutes ago; hopefully that is all that is needed. Ok, the git.pg.org was somewhat behind. It did fix it for me when I tested it locally. OK, that's good to know. So

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Petr Jelinek
On 02/04/15 22:22, Robert Haas wrote: On Thu, Apr 2, 2015 at 4:21 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 02/04/15 21:21, Robert Haas wrote: On Thu, Apr 2, 2015 at 2:07 PM, Robert Haas robertmh...@gmail.com wrote: I think this is really nice work, so I have committed this version. I

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Robert Haas
On Thu, Apr 2, 2015 at 4:21 PM, Petr Jelinek p...@2ndquadrant.com wrote: On 02/04/15 21:21, Robert Haas wrote: On Thu, Apr 2, 2015 at 2:07 PM, Robert Haas robertmh...@gmail.com wrote: I think this is really nice work, so I have committed this version. I made a few fairly minor changes,

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Andrew Gierth
Robert == Robert Haas robertmh...@gmail.com writes: Robert I think this is really nice work, so I have committed this Robert version. I made a few fairly minor changes, hopefully without Robert breaking anything in the process: Robert - I adjusted things for recent commits around

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Robert Haas
On Thu, Apr 2, 2015 at 2:07 PM, Robert Haas robertmh...@gmail.com wrote: I think this is really nice work, so I have committed this version. I made a few fairly minor changes, hopefully without breaking anything in the process: - I adjusted things for recent commits around

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 9:46 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 23, 2015 at 2:41 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter As I said, I don't really consider that my patch is a rewrite, Peter especially V4, which changes nothing substantive except

Re: [HACKERS] Abbreviated keys for Numeric

2015-04-02 Thread Robert Haas
On Tue, Mar 24, 2015 at 3:03 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: So here's the latest (and, hopefully, last) version: - adds diagnostic output from numeric_abbrev_abort using the trace_sort GUC - fixed Datum cs. uint32 issues in hash_uint32 - added a short comment

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-25 Thread Peter Geoghegan
On Tue, Mar 24, 2015 at 12:03 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: So here's the latest (and, hopefully, last) version: - adds diagnostic output from numeric_abbrev_abort using the trace_sort GUC - fixed Datum cs. uint32 issues in hash_uint32 - added a short comment

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-25 Thread Peter Geoghegan
On Wed, Mar 25, 2015 at 6:26 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter == Peter Geoghegan p...@heroku.com writes: Peter You still pointlessly check memtupcount here: Peter + if (memtupcount 1 || nss-input_count 1 || !nss-estimating) Peter + return false;

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-25 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter You still pointlessly check memtupcount here: Peter + if (memtupcount 1 || nss-input_count 1 || !nss-estimating) Peter + return false; It's in a register; the test is free. Peter This cast to void is unnecessary:

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-24 Thread k...@rice.edu
On Mon, Mar 23, 2015 at 09:41:40PM +, Andrew Gierth wrote: Peter == Peter Geoghegan p...@heroku.com writes: Peter As I said, I don't really consider that my patch is a rewrite, Peter especially V4, which changes nothing substantive except removing Peter 32-bit support. Well,

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Sat, Mar 21, 2015 at 2:58 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter I don't really buy it, either way. In what sense is a NULL value Peter ever abbreviated? It isn't. Whatever about the cost model, Peter that's the truth of the matter. There is always going to be a

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 12:20 PM, Robert Haas robertmh...@gmail.com wrote: Guys, can we please knock it off with the dueling patches? Peter, it's really not all that helpful to take somebody else's patch, rewrite it in a form that they may or may not agree with (even if it's just the

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 12:52 PM, Robert Haas robertmh...@gmail.com wrote: As + * with IEEE-754, we use an exponent without a sign (a 7-bit exponent + * without a sign). As to the beginning of this sentence, bringing IEEE-754 into this discussion doesn't clarify anything in my mind. I don't

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 12:04 PM, Robert Haas robertmh...@gmail.com wrote: In my opinion, Andrew's version is far clearer. Peter's version is full of jargon that I can't understand. I could probably figure it out with a few hours and a search engine, but that really shouldn't be necessary.

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 1:01 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Mar 22, 2015 at 1:01 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: The substance of the code is unchanged from my original patch. I didn't add diagnostic output to numeric_abbrev_abort, see my separate post

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 3:07 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 23, 2015 at 12:04 PM, Robert Haas robertmh...@gmail.com wrote: In my opinion, Andrew's version is far clearer. Peter's version is full of jargon that I can't understand. I could probably figure it out with a

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter As I said, I don't really consider that my patch is a rewrite, Peter especially V4, which changes nothing substantive except removing Peter 32-bit support. Well, that's a hell of an except. Here's my main arguments for why 32bit support

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 2:41 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter As I said, I don't really consider that my patch is a rewrite, Peter especially V4, which changes nothing substantive except removing Peter 32-bit support. Well, that's a hell of an except. I guess

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Sun, Mar 22, 2015 at 1:01 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: The substance of the code is unchanged from my original patch. I didn't add diagnostic output to numeric_abbrev_abort, see my separate post about the suggestion of a GUC for that. I don't think that V2 really

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 4:08 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: The earlier comment should make it clear that all the DEC_DIGITS != 4 support is historical. I didn't consider it appropriate to actually rip out all the #ifs; I simply tried to make it clear where the landmines

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 6:02 PM, Robert Haas robertmh...@gmail.com wrote: Well, not committing the patch at all would be even less invasive. But that's true of any patch, so I don't think being less invasive can be the prime goal. Of course it's usually better to be less invasive and get the

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter Your V3 has obsolete comments here: Peter + nss = palloc(sizeof(NumericSortSupport)); Peter + Peter + /* Peter + * palloc a buffer for handling unaligned packed values in addition to Peter + * the support struct Peter + */ Peter +

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 4:03 PM, Peter Geoghegan p...@heroku.com wrote: I must also admit that I am somewhat annoyed here, since Andrew has questioned essentially ever revision I've proposed to both of the sort support patches he wrote, and in a rather bellicose way. They were mostly very

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Peter Geoghegan
On Mon, Mar 23, 2015 at 5:47 PM, Robert Haas robertmh...@gmail.com wrote: I agree with you. Fewer and fewer people are running 32-bit systems these days, but there must surely be more people running 32-bit systems than there are running with DEC_DIGITS != 4. I think it's a stretch to say

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 8:54 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 23, 2015 at 5:47 PM, Robert Haas robertmh...@gmail.com wrote: I agree with you. Fewer and fewer people are running 32-bit systems these days, but there must surely be more people running 32-bit systems than

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 5:41 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter == Peter Geoghegan p...@heroku.com writes: Peter As I said, I don't really consider that my patch is a rewrite, Peter especially V4, which changes nothing substantive except removing Peter 32-bit

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-23 Thread Robert Haas
On Mon, Mar 23, 2015 at 9:12 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Mar 23, 2015 at 6:02 PM, Robert Haas robertmh...@gmail.com wrote: Well, not committing the patch at all would be even less invasive. But that's true of any patch, so I don't think being less invasive can be the

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Andrew Gierth
So here's a version 3 patch: 1. #ifdefism is reduced to a minimum by (a) desupporting values of NBASE other than 1, and (b) keeping the 32bit code, so that there is now no longer any question of abbreviation not being used (it always is). So the only #ifs in the code body (rather than

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Peter Geoghegan
On Sat, Mar 21, 2015 at 3:33 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Was there some reason why you added #include utils/memutils.h? Because I don't see anything in your patch that actually needs it. MaxAllocSize is defiined there. -- Peter Geoghegan -- Sent via pgsql-hackers

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Peter Geoghegan
On Fri, Mar 20, 2015 at 11:58 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Comparisons between nulls and nulls, or between nulls and non-nulls, are cheap; only comparisons between non-nulls and non-nulls can be expensive. The purpose of abbreviation is to replace expensive comparisons

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter I don't think it's that simple. The actual point of abbreviation Peter is to amortize the total cost of performing O(n log n) Peter comparisons (the average case, which is usually assumed by Peter infrastructure like sortsupport), by

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Was there some reason why you added #include utils/memutils.h? Because I don't see anything in your patch that actually needs it. Peter MaxAllocSize is defiined there. So it is. (That seems to me to be another mistake along the same lines

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-22 Thread Peter Geoghegan
On Sun, Mar 22, 2015 at 6:48 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Your version would have aborted abbrevation on that second query, thus losing a 3 second speedup. How on earth is that supposed to be justified? It's not like there's any realistically possible case where your

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-21 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: This is simply wrong. The reason why the cost model (in my version) tracks non-null values by having its own counter is precisely BECAUSE the passed-in memtupcount includes nulls, and therefore the code will UNDERESTIMATE the fraction of

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-21 Thread Andrew Gierth
Was there some reason why you added #include utils/memutils.h? Because I don't see anything in your patch that actually needs it. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-20 Thread Andrew Gierth
Peter == Peter Geoghegan p...@heroku.com writes: Peter Attached is a revision of this patch, that I'm calling v2. What Peter do you think, Andrew? No. is I think the best summary of my response. I strongly suggest whichever committer ends up looking at this consider my original version

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-20 Thread Peter Geoghegan
Attached is a revision of this patch, that I'm calling v2. What do you think, Andrew? I've cut the int32 representation/alternative !USE_FLOAT8_BYVAL encoding scheme entirely, which basically means that 32-bit systems don't get to have this optimization. 64-bit systems have been commonplace now

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-20 Thread Peter Geoghegan
On Fri, Mar 20, 2015 at 7:10 PM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Peter Other than that, I've tried to keep things closer to the text Peter opclass. For example, the cost model now has a few debugging Peter traces (disabled by default). I have altered the ad-hoc cost Peter

Re: [HACKERS] Abbreviated keys for Numeric

2015-03-04 Thread Robert Haas
On Mon, Feb 23, 2015 at 5:59 AM, Andrew Gierth and...@tao11.riddles.org.uk wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas Interesting, but I think Gavin was asking about how much Tomas variation was there for each tested case (e.g. query executed on Tomas the same

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-24 Thread Tomas Vondra
On 24.2.2015 05:09, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas I believe the small regressions (1-10%) for small data sets, Tomas might be caused by this 'random padding' effect, because that's Tomas probably where L1/L2 cache is most important.

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-23 Thread Andrew Gierth
Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas Interesting, but I think Gavin was asking about how much Tomas variation was there for each tested case (e.g. query executed on Tomas the same code / dataset). And in those cases the padding / Tomas alignment won't change, and

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-23 Thread Tomas Vondra
Hi, On 23.2.2015 11:59, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas Interesting, but I think Gavin was asking about how much Tomas variation was there for each tested case (e.g. query executed on Tomas the same code / dataset). And in those cases

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-23 Thread Andrew Gierth
Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Tomas I believe the small regressions (1-10%) for small data sets, Tomas might be caused by this 'random padding' effect, because that's Tomas probably where L1/L2 cache is most important. For large datasets Tomas the caches are

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Andrew Gierth
Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin What are the standard deviations? Gavin Do the arithmetic means change much if you exclude the 2 fastest Gavin 2 slowest? Gavin How do the arithmetic means compare to their respective medians? Gavin Essentially, how

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Gavin Flower
On 22/02/15 22:59, Andrew Gierth wrote: Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin What are the standard deviations? Gavin Do the arithmetic means change much if you exclude the 2 fastest Gavin 2 slowest? Gavin How do the arithmetic means compare to their

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Tomas Vondra
On 22.2.2015 10:59, Andrew Gierth wrote: Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin Essentially, how consistent are the results, or how great is the Gavin noise? There may be better indicators than the ones I've Gavin suggested above. This is all rather missing

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Although I cannot easily explain the disparity in performance between 1M and 5M sized sets for this query: select count(distinct randtxt) from stuff_text You did make sure that the queries didn't spill to

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 23:09, Peter Geoghegan wrote: On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Although I cannot easily explain the disparity in performance between 1M and 5M sized sets for this query: select count(distinct randtxt) from stuff_text You did

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 2:09 PM, Peter Geoghegan p...@heroku.com wrote: I can recreate the problem you see with text sort regressions. Abbreviation is aborted for the case in question, unsurprisingly, and fairly far in. With that many tuples, the idea of taking abbreviated cardinality as a

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
Hi Gavin, On 21.2.2015 06:35, Gavin Flower wrote: On 21/02/15 18:18, Tomas Vondra wrote: OK, so I've repeated the benchmarks with both patches applied, and I think the results are interesting. I extended the benchmark a bit - see the SQL script attached. 1) multiple queries

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's actually slower than current master :-( That's odd. I have a hard time

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 19:57, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's actually slower than current

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 20:33, Peter Geoghegan wrote: On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan p...@heroku.com That's odd. I have a hard time thinking of why the datum sort patch could be at fault, though. Oh, wait. For queries like this, which I now see in your spreadsheet: select *

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 1:33 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: For example with the same percentile_disc() test as in the other thread: create table stuff as select random()::numeric as randnum from generate_series(1,100); analyze

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 00:14, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 1:33 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: For example with the same percentile_disc() test as in the other thread: create table stuff as select random()::numeric as randnum from generate_series(1,100);

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 4:42 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Isn't this patch about adding abbreviated keys for Numeric data type? That's how I understood it, and looking into numeric_sortsup.patch seems to confirm that. There's another patch for Datum, but that's a

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 01:45, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 4:42 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Isn't this patch about adding abbreviated keys for Numeric data type? That's how I understood it, and looking into numeric_sortsup.patch seems to confirm that.

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Tomas Vondra
On 21.2.2015 02:00, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your test case is invalid for that reason. Tomas What do you mean by 'Datum sort

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Andrew Gierth
Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your test case is invalid for that reason. Tomas What do you mean by 'Datum sort case'? A case where the code path goes via

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 4:11 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: So you're testing both the patches (numeric + datum tuplesort) at the same time? No, I was just testing two similar patches separately. I.e. master vs. each patch separately. Well, you're sorting numeric here,

Re: [HACKERS] Abbreviated keys for Numeric (was: Re: B-Tree support function number 3 (strxfrm() optimization))

2015-02-20 Thread Tomas Vondra
On 21.2.2015 01:17, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 4:11 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: So you're testing both the patches (numeric + datum tuplesort) at the same time? No, I was just testing two similar patches separately. I.e. master vs. each patch

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Tomas Vondra
Hi, On 21.2.2015 02:06, Tomas Vondra wrote: On 21.2.2015 02:00, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your test case is invalid for that reason.

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-20 Thread Gavin Flower
On 21/02/15 18:18, Tomas Vondra wrote: Hi, On 21.2.2015 02:06, Tomas Vondra wrote: On 21.2.2015 02:00, Andrew Gierth wrote: Tomas == Tomas Vondra tomas.von...@2ndquadrant.com writes: Right...so don't test a datum sort case, since that isn't supported at all in the master branch. Your