Re: [HACKERS] Performance optimization of btree binary search

2013-12-09 Thread Peter Geoghegan
On Fri, Dec 6, 2013 at 4:53 PM, Peter Geoghegan wrote: > I had considered that something like Intel Speedstep technology had a > role here, but I'm pretty sure it steps up very aggressively when > things are CPU bound - I tested that against a Core 2 Duo desktop a > couple of years back, where it

Re: [HACKERS] Performance optimization of btree binary search

2013-12-06 Thread Peter Geoghegan
On Thu, Dec 5, 2013 at 2:19 AM, Peter Geoghegan wrote: > I did a relatively short variant 'insert' pgbench-tools benchmark, > with a serial primary index created on the pgbench_history table. > Results: > > http://postgres-benchmarks.s3-website-us-east-1.amazonaws.com/insert/ I saw something toda

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Eisentraut
On Thu, 2013-12-05 at 15:29 +0200, Heikki Linnakangas wrote: > It happens to me about 75% of the time when I write a new C function. > These days, I usually realize it pretty quickly. > > I wonder how easy it would be to make the compiler produce a warning > about it. Or issue a warning in Postg

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund writes: > On 2013-12-05 10:34:16 -0500, Tom Lane wrote: >> In any case, the number of bugs I can remember that such a thing >> would've prevented is negligible. > Cases talked about upthread, where a plain datatype is returned as a > Datum instead of using FooGetDatum() and the reve

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:34:16 -0500, Tom Lane wrote: > Andres Freund writes: > > I was actually thinking about making Datum (and some other types we > > have) structs or unions. Currently it's far, far to easy to mix them. We > > throw > > away pretty much all of the little typesafety C has by typedef'i

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund writes: > I was actually thinking about making Datum (and some other types we > have) structs or unions. Currently it's far, far to easy to mix them. We throw > away pretty much all of the little typesafety C has by typedef'ing them > to integral types with lots of autocasting behavi

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 10:02:56 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2013-12-05 08:58:55 -0500, Tom Lane wrote: > >> I'm a bit worried that somebody, particularly third-party code, > >> might've sloppily written "return foo" in a V1 function when "return > >> Int32GetDatum(foo)" would be c

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund writes: > On 2013-12-05 08:58:55 -0500, Tom Lane wrote: >> I'm a bit worried that somebody, particularly third-party code, >> might've sloppily written "return foo" in a V1 function when "return >> Int32GetDatum(foo)" would be correct. In that case, the resultant Datum >> might have

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 08:58:55 -0500, Tom Lane wrote: > Andres Freund writes: > > I don't think we can get rid of that dance in record_image_eq - it very > > well could used on records originally generated when those bits haven't > > been guaranteed to be zeroed. > > No, you're failing to think about the

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund writes: > On 2013-12-04 18:48:44 -0500, Robert Haas wrote: >> And record_image_eq does a rather elaborate dance around here, calling >> the appropriate GET_x_BYTES macro depending on the type-width. If we >> can really count on the high-order bits to be zero, that's all >> completel

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 07:30 AM, Tom Lane wrote: Peter Eisentraut writes: On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I think it probably c

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-04 18:48:44 -0500, Robert Haas wrote: > * When a type narrower than Datum is stored in a Datum, we place it in the > * low-order bits and are careful that the DatumGetXXX macro for it discards > * the unused high-order bits (as opposed to, say, assuming they are zero). > * This is ne

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan wrote: > I'm also curious about the impact on insertion into primary key > indexes. Presently, we hold an exclusive buffer lock for the duration > of a couple of operations when checkUnique != UNIQUE_CHECK_NO. > _bt_binsrch() is one such operation. T

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Eisentraut writes: > On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: >> Lazy people? I'm not in a hurry to drop it; it's not costing us much to >> just sit there, other than in this connection which we see how to fix. > Actually, I think it probably costs a fair portion of extension aut

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Eisentraut
On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: > Lazy people? I'm not in a hurry to drop it; it's not costing us much to > just sit there, other than in this connection which we see how to fix. Actually, I think it probably costs a fair portion of extension authors when their initial code cra

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan wrote: > I'm also curious about the impact on insertion into primary key > indexes. Presently, we hold an exclusive buffer lock for the duration > of a couple of operations when checkUnique != UNIQUE_CHECK_NO. > _bt_binsrch() is one such operation. T

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 12:58 PM, Peter Geoghegan wrote: > I'm kind of > curious as to what this benchmark would like like on a server with > many more cores. I'm also curious about the impact on insertion into primary key indexes. Presently, we hold an exclusive buffer lock for the duration of a

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Eisentraut writes: > On Wed, 2013-12-04 at 19:45 -0500, Robert Haas wrote: >> On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane wrote: >>> Yeah, that's another thing we could simplify if we fixed this problem >>> at the source. I think these decisions date from a time when we still >>> cared about

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Eisentraut
On Wed, 2013-12-04 at 19:45 -0500, Robert Haas wrote: > On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane wrote: > > Yeah, that's another thing we could simplify if we fixed this problem > > at the source. I think these decisions date from a time when we still > > cared about the speed of fmgr_oldstyle. >

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 6:56 PM, Tom Lane wrote: > Robert Haas writes: >> Hmm. And yet, there's this: > >> * When a type narrower than Datum is stored in a Datum, we place it in the >> * low-order bits and are careful that the DatumGetXXX macro for it discards >> * the unused high-order bits (

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
I wrote: > Yeah, that's another thing we could simplify if we fixed this problem > at the source. I think these decisions date from a time when we still > cared about the speed of fmgr_oldstyle. BTW, the text you're quoting is from 2007, but it's just documenting behavior that's mostly a lot olde

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Robert Haas writes: > Hmm. And yet, there's this: > * When a type narrower than Datum is stored in a Datum, we place it in the > * low-order bits and are careful that the DatumGetXXX macro for it discards > * the unused high-order bits (as opposed to, say, assuming they are zero). > * This i

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 6:33 PM, Peter Geoghegan wrote: > On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas wrote: >> Yeah, I think if we can make something like this work, it would be >> neat-o. Getting this working for int4 would be a good win, as Peter >> says, but getting it working for both int4 a

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas wrote: >> Yeah, I think if we can make something like this work, it would be >> neat-o. Getting this working for int4 would be a good win, as Peter >> says, but getting it working for both int4 and int8 with the same code >> w

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 1:59 PM, Robert Haas wrote: > Yeah, I think if we can make something like this work, it would be > neat-o. Getting this working for int4 would be a good win, as Peter > says, but getting it working for both int4 and int8 with the same code > would be a significantly better

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Robert Haas
On Wed, Dec 4, 2013 at 4:28 PM, Tom Lane wrote: > Peter Geoghegan writes: >> I guess I could write a proper patch to have code setting up a scankey >> also set a flag that indicated that it was acceptable to assume that >> the special built-in comparator would do fine. ... >> I'd be happy with a

Re: [HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Tom Lane
Peter Geoghegan writes: > I guess I could write a proper patch to have code setting up a scankey > also set a flag that indicated that it was acceptable to assume that > the special built-in comparator would do fine. ... > I'd be happy with a scheme with only one built-in comparator, and > allowed

[HACKERS] Performance optimization of btree binary search

2013-12-04 Thread Peter Geoghegan
Having nothing better to do over the holiday weekend, I decided to pursue a number of ideas for improving performance that I thought about a long time ago. These include: * Pre-fetching list node pointers. This looks to be moderately promising, but I'm certainly not going to be the one to land it,