Re: [HACKERS] WALInsertLock contention

2011-06-09 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 11:30 PM, Merlin Moncure mmonc...@gmail.com wrote: The heap pages that have been marked this way may or may not have to be off limits from the backend other than the one that did the marking, and if they have to be off limits logically, there may be no realistic path to

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 1:59 AM, Merlin Moncure mmonc...@gmail.com wrote: There's probably an obvious explanation that I'm not seeing, ... Yep. :-) but if you're not delegating the work of writing the buffers out to someone else, why do you need to lock the per backend buffer at all?  That

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 7:44 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 1:59 AM, Merlin Moncure mmonc...@gmail.com wrote: There's probably an obvious explanation that I'm not seeing, ... Yep.  :-) but if you're not delegating the work of writing the buffers out to

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 10:18 AM, Merlin Moncure mmonc...@gmail.com wrote: One point i'm missing though.  Getting back to your original idea, how does writing to shmem prevent you from having to keep buffers pinned? I'm reading your comment here: Those buffers are stamped with a fake LSN that

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Jim Nasby
On Jun 8, 2011, at 10:15 AM, Robert Haas wrote: That suggests to me that you have to keep them pinned anyways. I'm still a bit fuzzy on how the per-backend buffers being in shm conveys any advantage. IOW, (trying not to be obtuse) under what circumstances would backend A want to read from or

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 6:49 PM, Jim Nasby j...@nasby.net wrote: If backend A needs to evict a buffer with a fake LSN, it can go find the WAL that needs to be serialized, do that, flush WAL, and then evict the buffer. Isn't the only time that you'd need to evict if you ran out of buffers?

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 10:21 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 6:49 PM, Jim Nasby j...@nasby.net wrote: If backend A needs to evict a buffer with a fake LSN, it can go find the WAL that needs to be serialized, do that, flush WAL, and then evict the buffer.

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Robert Haas
On Wed, Jun 8, 2011 at 11:20 PM, Merlin Moncure mmonc...@gmail.com wrote: You're probably right.  I think though there is enough hypothetical upside to the private buffer case that it should be attempted just to see what breaks. The major tricky bit is dealing with the new pin/unpin mechanics.

Re: [HACKERS] WALInsertLock contention

2011-06-08 Thread Merlin Moncure
On Wed, Jun 8, 2011 at 11:27 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Jun 8, 2011 at 11:20 PM, Merlin Moncure mmonc...@gmail.com wrote: You're probably right.  I think though there is enough hypothetical upside to the private buffer case that it should be attempted just to see what

Re: [HACKERS] WALInsertLock contention

2011-06-07 Thread Merlin Moncure
On Wed, Feb 16, 2011 at 11:02 PM, Robert Haas robertmh...@gmail.com wrote: I've been thinking about the problem of $SUBJECT, and while I know it's too early to think seriously about any 9.2 development, I want to get my thoughts down in writing while they're fresh in my head. It seems to me

[HACKERS] WALInsertLock contention

2011-02-16 Thread Robert Haas
I've been thinking about the problem of $SUBJECT, and while I know it's too early to think seriously about any 9.2 development, I want to get my thoughts down in writing while they're fresh in my head. It seems to me that there are two basic approaches to this problem. We could either split up

Re: [HACKERS] WALInsertLock contention

2011-02-16 Thread Tatsuo Ishii
I've been thinking about the problem of $SUBJECT, and while I know it's too early to think seriously about any 9.2 development, I want to get my thoughts down in writing while they're fresh in my head. It seems to me that there are two basic approaches to this problem. We could either split

Re: [HACKERS] WALInsertLock contention

2011-02-16 Thread Robert Haas
On Wed, Feb 16, 2011 at 11:13 PM, Tatsuo Ishii is...@postgresql.org wrote: I vaguely recall that UNISYS used to present patches to reduce the WAL buffer lock contention and enhanced the CPU scalability limit from 12 to 16 or so(if my memory serves). Your second idea is somewhat related to the

Re: [HACKERS] WALInsertLock contention

2011-02-16 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Wed, Feb 16, 2011 at 11:13 PM, Tatsuo Ishii is...@postgresql.org wrote: I vaguely recall that UNISYS used to present patches to reduce the WAL buffer lock contention and enhanced the CPU scalability limit from 12 to 16 or so(if my memory

Re: [HACKERS] WALInsertLock contention

2011-02-16 Thread Tatsuo Ishii
Not sure. Do you have a link to the archives, or any idea when this discussion occurred/what the subject line was? They presented at PgCon a couple of years in a row, iirc.. http://www.pgcon.org/2007/schedule/events/16.en.html Yes, this one. On page 18, they talked about their customized