Re: [HACKERS] Page-at-a-time Locking Considerations

2008-03-23 Thread Simon Riggs
On Sat, 2008-03-22 at 20:37 -0400, Bruce Momjian wrote: > With no concrete patch or performance numbers, this thread has been > removed from the patches queue. I agree since there is no patch. However, I think recent performance reports around the cost of visibility checks such as "Very slow seq

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-03-22 Thread Bruce Momjian
With no concrete patch or performance numbers, this thread has been removed from the patches queue. --- Simon Riggs wrote: > > In heapgetpage() we hold the buffer locked while we look for visible > tuples. That works well i

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-07 Thread Zdenek Kotala
Gregory Stark napsal(a): "Bruce Momjian" <[EMAIL PROTECTED]> writes: You can flush a pages by msync() function which writes dirty pages on disk. I don't see any other problem. Then you need to learn more. The side of the problem that is hard to fix is that sometimes we need to prevent pages f

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-07 Thread Simon Riggs
On Mon, 2008-02-04 at 20:54 +, Simon Riggs wrote: > On Mon, 2008-02-04 at 20:03 +, Gregory Stark wrote: > > > I wonder how hard it would be to shove the clog into regular shared > > memory pages and let the clock sweep take care of adjusting the > > percentage of shared mem allocated to th

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-07 Thread Gregory Stark
"Bruce Momjian" <[EMAIL PROTECTED]> writes: >> >> You can flush a pages by msync() function which writes dirty pages on >> >> disk. I don't see any other problem. >> > >> > Then you need to learn more. The side of the problem that is hard to >> > fix is that sometimes we need to prevent pages f

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Bruce Momjian
Zdenek Kotala wrote: > Tom Lane wrote: > > Zdenek Kotala <[EMAIL PROTECTED]> writes: > >> Tom Lane wrote: > >>> If you only got 2% out of it, it's not even worth thinking about how to > >>> fix the serious bugs that approach would create (primarily, lack of > >>> control over when pages can get flu

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Gregory Stark
"Zdenek Kotala" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Zdenek Kotala <[EMAIL PROTECTED]> writes: >>> Tom Lane wrote: If you only got 2% out of it, it's not even worth thinking about how to fix the serious bugs that approach would create (primarily, lack of control over whe

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Zdenek Kotala
Tom Lane wrote: Zdenek Kotala <[EMAIL PROTECTED]> writes: Tom Lane wrote: If you only got 2% out of it, it's not even worth thinking about how to fix the serious bugs that approach would create (primarily, lack of control over when pages can get flushed to disk). You can flush a pages by msyn

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Tom Lane
Zdenek Kotala <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> If you only got 2% out of it, it's not even worth thinking about how to >> fix the serious bugs that approach would create (primarily, lack of >> control over when pages can get flushed to disk). > You can flush a pages by msync() func

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Zdenek Kotala
Tom Lane wrote: Zdenek Kotala <[EMAIL PROTECTED]> writes: I tried to use memory mapped files (mmap) for clog and I think it should be also possible way. I got about 2% better performance, but it needs more testing. If you only got 2% out of it, it's not even worth thinking about how to fix the

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Tom Lane
Zdenek Kotala <[EMAIL PROTECTED]> writes: > I tried to use memory mapped files (mmap) for clog and I think it should be > also > possible way. I got about 2% better performance, but it needs more testing. If you only got 2% out of it, it's not even worth thinking about how to fix the serious bug

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-06 Thread Zdenek Kotala
Gregory Stark napsal(a): "Simon Riggs" <[EMAIL PROTECTED]> writes: I wonder how hard it would be to shove the clog into regular shared memory pages and let the clock sweep take care of adjusting the percentage of shared mem allocated to the clog versus data pages. I tried to use memory mapp

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-05 Thread Simon Riggs
On Mon, 2008-02-04 at 18:08 -0300, Alvaro Herrera wrote: > Simon Riggs wrote: > > On Mon, 2008-02-04 at 20:03 +, Gregory Stark wrote: > > > > > I wonder how hard it would be to shove the clog into regular shared > > > memory pages and let the clock sweep take care of adjusting the > > > percen

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Alvaro Herrera
Tom Lane wrote: > > Gregory Stark wrote: > >> I wonder how hard it would be to shove the clog into regular shared memory > >> pages and let the clock sweep take care of adjusting the percentage of > >> shared > >> mem allocated to the clog versus data pages. > My recollection is that we didn't d

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Heikki Linnakangas
Alvaro Herrera wrote: Gregory Stark wrote: I wonder how hard it would be to shove the clog into regular shared memory pages and let the clock sweep take care of adjusting the percentage of shared mem allocated to the clog versus data pages. Hmm, this is an interesting idea. I wonder what wou

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Alvaro Herrera
Simon Riggs wrote: > On Mon, 2008-02-04 at 20:03 +, Gregory Stark wrote: > > > I wonder how hard it would be to shove the clog into regular shared > > memory pages and let the clock sweep take care of adjusting the > > percentage of shared mem allocated to the clog versus data pages. > > Ther

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Gregory Stark wrote: >> I wonder how hard it would be to shove the clog into regular shared memory >> pages and let the clock sweep take care of adjusting the percentage of shared >> mem allocated to the clog versus data pages. > Hmm, this is an interes

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Simon Riggs
On Mon, 2008-02-04 at 20:03 +, Gregory Stark wrote: > I wonder how hard it would be to shove the clog into regular shared > memory pages and let the clock sweep take care of adjusting the > percentage of shared mem allocated to the clog versus data pages. There is a reason that's not been don

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Alvaro Herrera
Gregory Stark wrote: > I wonder how hard it would be to shove the clog into regular shared memory > pages and let the clock sweep take care of adjusting the percentage of shared > mem allocated to the clog versus data pages. Hmm, this is an interesting idea. I wonder what would happen if we let

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Gregory Stark
"Simon Riggs" <[EMAIL PROTECTED]> writes: > We still have a higher than desirable variability in response times and > I'm looking at possible causes. I agree we have a problem with this. My feeling is that the problems have more to do with higher level things like stats being toasted, or checkpoi

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Simon Riggs
On Mon, 2008-02-04 at 13:27 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > In heapgetpage() we hold the buffer locked while we look for visible > > tuples. > > It's a share lock though. Which conflicts with write locks. > Do you have any direct proof that this > behavior

Re: [HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > In heapgetpage() we hold the buffer locked while we look for visible > tuples. It's a share lock though. Do you have any direct proof that this behavior is as nasty as you claim? regards, tom lane ---(end

[HACKERS] Page-at-a-time Locking Considerations

2008-02-04 Thread Simon Riggs
In heapgetpage() we hold the buffer locked while we look for visible tuples. That works well in most cases since the visibility check is fast if we have status bits set. If we don't have visibility bits set we have to do things like scan the snapshot and confirm things via clog lookups. All of tha