Re: [HACKERS] Further reduction of bufmgr lock contention

2006-08-26 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Is this being kept for 8.3? No, it was committed a month ago. regards, tom lane ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-08-25 Thread Bruce Momjian
Is this being kept for 8.3? --- Tom Lane wrote: > A couple months ago we were discussing partitioning the buffer mapping > table so as to reduce contention for the BufMappingLock. The discussion > stalled after I complained

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-07-21 Thread Simon Riggs
On Thu, 2006-07-20 at 15:41 -0400, Tom Lane wrote: > Usage of a partitioned hash table would then be like > > compute hashtable lookup key; > entryhashcode = calc_hash(lookup key); > partitionnumber = entryhashcode % NumPartitions; > LWLockAcquire(PartitionLock[partitionnu

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-07-20 Thread Tom Lane
A couple months ago we were discussing partitioning the buffer mapping table so as to reduce contention for the BufMappingLock. The discussion stalled after I complained about the uncertainty of shared memory usage arising from having a separate hashtable for each partition (since different number

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-29 Thread Bruce Momjian
Is this a TODO? --- Tom Lane wrote: > Josh Berkus writes: > > BTW, we're going to be testing this patch on Sun Niagara servers. What's > > the outstanding bug with it? I don't quite follow. > > It's not acceptable as-

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-24 Thread Tom Lane
Josh Berkus writes: > BTW, we're going to be testing this patch on Sun Niagara servers. What's > the outstanding bug with it? I don't quite follow. It's not acceptable as-is because of the risk of running out of shared memory for hashtable entries. In the existing code, there's a clear uppe

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-24 Thread Josh Berkus
Tom, BTW, we're going to be testing this patch on Sun Niagara servers. What's the outstanding bug with it? I don't quite follow. I think I can get some of the Sun MDEs to take a stab at it if I can understand the issue. Links ok if maybe I've not found part of this thread in the archives.

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-24 Thread Gavin Hamill
Tom Lane wrote: We're certainly not putting any such thing into 8.1.*. The proposed patch for 8.2 is stalled ATM because of the problem of not having a predictable size for the per-partition hash tables. Fixed-size shared memory is a harsh mistress :-( Fair enough :) Just wanted to ascertai

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-23 Thread Tom Lane
Gavin Hamill <[EMAIL PROTECTED]> writes: > It's been about a month since the last activity on bufmgr as documented > on the hackers list and I was just concerned that this issue had been > filed as an interesting toy at the time, but now left for the circular > filing cabinet :) > Tom + Simon w

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-05-23 Thread Gavin Hamill
Tom Lane wrote: I've been looking into Gavin Hamill's recent report of poor performance with PG 8.1 on an 8-way IBM PPC64 box. [...] Hullo again :) I'm unfamiliar with postgres development practices, so this is more a request for information than anything else. It's been about a month sinc

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-24 Thread Mark Wong
Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: On Fri, 2006-04-21 at 13:01 -0400, Tom Lane wrote: I've been looking into Gavin Hamill's recent report of poor performance with PG 8.1 on an 8-way IBM PPC64 box. Keep in mind that Gavin's 8-way turns back into a pumpkin on Monday :-(

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-24 Thread Gavin Hamill
On Fri, 21 Apr 2006 17:38:01 -0400 Tom Lane <[EMAIL PROTECTED]> wrote: > I believe the particular test case being looked at here is read-only > (Gavin, is that correct?) Yes - I made sure the devels made it readonly so I could farm search requests out to Slony-replicated machines (ended up runnin

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-24 Thread Qingqing Zhou
"Tom Lane" <[EMAIL PROTECTED]> wrote > > The thing that's nagging at me at the moment is the realization that a > partitioned hashtable will eat more shared memory than a single > hashtable. It wasn't that long ago that we had to do some hacking to > ensure that the buffer hashtable couldn't run

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > lmgr partitioning uses either 4 or 16, restricted by the hash function, > for various reasons. I see no similar restriction on using a hash > function here - we could equally well use range partitioning. I don't really see any difference at all between the

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Simon Riggs
On Fri, 2006-04-21 at 17:38 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > The earlier lmgr lock partitioning had a hard-coded number of > > partitions, which was sensible because of the reduced likelihood of > > effectiveness beyond a certain number of partitions. That doesn

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Fri, 2006-04-21 at 13:01 -0400, Tom Lane wrote: >> I've been looking into Gavin Hamill's recent report of poor performance >> with PG 8.1 on an 8-way IBM PPC64 box. > BufMappingLock contention can be made worse by a poorly tuned bgwriter > or if the ca

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Simon Riggs
On Fri, 2006-04-21 at 13:01 -0400, Tom Lane wrote: > I've been looking into Gavin Hamill's recent report of poor performance > with PG 8.1 on an 8-way IBM PPC64 box. Ah good. > Instrumenting LWLockAcquire (with a patch I had developed last fall, > but just now got around to cleaning up and commi

Re: [HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Jonah H. Harris
On 4/21/06, Tom Lane <[EMAIL PROTECTED]> wrote: > I've been looking into Gavin Hamill's recent report of poor performance > with PG 8.1 on an 8-way IBM PPC64 box. We have recently encountered some odd performance with 8.2dev on a 16-way Opteron. In the next few days we'll look into it and see if

[HACKERS] Further reduction of bufmgr lock contention

2006-04-21 Thread Tom Lane
I've been looking into Gavin Hamill's recent report of poor performance with PG 8.1 on an 8-way IBM PPC64 box. strace'ing backends shows a lot of semop() calls, indicating blocking at the LWLock or lmgr-lock levels, but not a lot of select() delays, suggesting we don't have too much of a problem a