Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-22 Thread Simon Riggs
On Mon, 2005-02-21 at 18:45 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: ...but do you agree with my comments on the lack of scalability in cache miss situations? No. Grabbing a lock during a cache miss is the least of your worries; you're going to do I/O, or at least a

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-21 Thread Simon Riggs
My understanding from this is: If we have a buffer cache hit ratio of 93%, then we should expect: - 93% of buffer requests to require only shared BufMappingLocks - 7% of buffer requests would require an exclusive BufFreelistLock then an exclusive BufMappingLock. That seems like an improvement

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: [I'm assuming that there are no system-wide locks held across I/Os, that bit seems a bit unclear from the description] That's always been true and still is, so I didn't dwell on it. Only a per-buffer lock is held while doing either input or output.

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-21 Thread Simon Riggs
On Mon, 2005-02-21 at 18:01 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: [I'm assuming that there are no system-wide locks held across I/Os, that bit seems a bit unclear from the description] That's always been true and still is, so I didn't dwell on it. Only a per-buffer

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: ...but do you agree with my comments on the lack of scalability in cache miss situations? No. Grabbing a lock during a cache miss is the least of your worries; you're going to do I/O, or at least a kernel call, so it hardly matters as long as you're not

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-21 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: This design seems to be a clear improvement on the current design. I am still encouraged that the freelist structures should be subdivided into many smaller pieces, thereby producing finer grained locks (the earlier bufferpools proposal). As I already

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-20 Thread Bruce Momjian
Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The advantage of using a counter instead of a simple active bit is that buffers that are (or have been) used heavily will be able to go through several sweeps of the clock before being freed. Infrequently used buffers (such as those

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-17 Thread Richard Huxton
Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The advantage of using a counter instead of a simple active bit is that buffers that are (or have been) used heavily will be able to go through several sweeps of the clock before being freed. Infrequently used buffers (such as those from a

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-17 Thread Zeugswetter Andreas DAZ SD
Would there be any value in incrementing by 2 for index accesses and 1 for seq-scans/vacuums? Actually, it should probably be a ratio based on random_page_cost shouldn't it? What happens with very small hot tables that are only a few pages and thus have no index defined. I think it would

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-16 Thread Jim C. Nasby
On Sun, Feb 13, 2005 at 06:56:47PM -0500, Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: One thing I realized quickly is that there is no natural way in a clock algorithm to discourage VACUUM from blowing out the cache. I came up with a slightly ugly idea

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-16 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: The advantage of using a counter instead of a simple active bit is that buffers that are (or have been) used heavily will be able to go through several sweeps of the clock before being freed. Infrequently used buffers (such as those from a vacuum or seq.

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-16 Thread Kenneth Marshall
On Wed, Feb 16, 2005 at 12:33:38PM -0500, Tom Lane wrote: Jim C. Nasby [EMAIL PROTECTED] writes: The advantage of using a counter instead of a simple active bit is that buffers that are (or have been) used heavily will be able to go through several sweeps of the clock before being freed.

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-16 Thread Jim C. Nasby
On Wed, Feb 16, 2005 at 11:42:11AM -0600, Kenneth Marshall wrote: I have seen this algorithm described as a more generalized clock type algorithm. As the size of the counter increases, up to the number of buffers, the clock algorithm becomes LRU. One bit is the lightest weight approximation.

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-14 Thread Sailesh Krishnamurthy
Tom == Tom Lane [EMAIL PROTECTED] writes: Tom and changing the buf_table hash table. The only common Tom operation that needs exclusive lock is reading in a page that Tom was not in shared buffers already, which will require at Tom least a kernel call and usually a wait for I/O,

[HACKERS] Design notes for BufMgrLock rewrite

2005-02-13 Thread Tom Lane
I'm working on an experimental patch to break up the BufMgrLock along the lines we discussed a few days ago --- in particular, using a clock sweep algorithm instead of LRU lists for the buffer replacement strategy. I started by writing up some design notes, which are attached for review in case

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-13 Thread Bruce Momjian
Tom Lane wrote: I'm working on an experimental patch to break up the BufMgrLock along the lines we discussed a few days ago --- in particular, using a clock sweep algorithm instead of LRU lists for the buffer replacement strategy. I started by writing up some design notes, which are attached

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-13 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: One thing I realized quickly is that there is no natural way in a clock algorithm to discourage VACUUM from blowing out the cache. I came up with a slightly ugly idea that's described below. Can anyone do better? Uh, is the clock

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-13 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: One thing I realized quickly is that there is no natural way in a clock algorithm to discourage VACUUM from blowing out the cache. I came up with a slightly ugly idea that's described below. Can anyone do

Re: [HACKERS] Design notes for BufMgrLock rewrite

2005-02-13 Thread Greg Stark
Bruce Momjian pgman@candle.pha.pa.us writes: Tom Lane wrote: One thing I realized quickly is that there is no natural way in a clock algorithm to discourage VACUUM from blowing out the cache. I came up with a slightly ugly idea that's described below. Can anyone do better? Uh, is