Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Robert Haas
On Sun, Nov 14, 2010 at 8:31 PM, Robert Haas robertmh...@gmail.com wrote: On Sun, Nov 14, 2010 at 7:19 PM, Greg Smith g...@2ndquadrant.com wrote: But if this is generating a lot of log data or adding a lot of overhead, then you have bigger problems anyway: +               elog(DEBUG1, Unable

Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Jeff Janes
On Sun, Nov 14, 2010 at 2:07 PM, Greg Smith g...@2ndquadrant.com wrote: The attached patch adds a new field to pg_stat_bgwriter, counting the number of times backends execute their own fsync calls.  Normally, when a backend needs to fsync data, it passes a request to the background writer,

Re: [HACKERS] Count backend self-sync calls

2010-11-15 Thread Greg Smith
Jeff Janes wrote: Do you know where this competition is happening? Is it on the platters, or is it in the hard drive write cache (I thought high-end hardware had tagged writes to avoid that), or in the kernel? Kernel. Linux systems with lots of memory will happily queue up gigabytes of

[HACKERS] Count backend self-sync calls

2010-11-14 Thread Greg Smith
The attached patch adds a new field to pg_stat_bgwriter, counting the number of times backends execute their own fsync calls. Normally, when a backend needs to fsync data, it passes a request to the background writer, which then absorbs the call into its own queue of work to do. However,

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 5:07 PM, Greg Smith g...@2ndquadrant.com wrote: The patch also adds some logging to the innards involved here, to help with understanding problems in this area.  I don't think that should be in the version committed as is.  May want to drop the logging level or make it

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: With those changes, I think this is committable, and will do so, barring objections. Obey message style guidelines, please, especially if you're going to promote any of those to ereports. regards, tom lane -- Sent via

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 6:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: With those changes, I think this is committable, and will do so, barring objections. Obey message style guidelines, please, especially if you're going to promote any of those to

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Greg Smith
Robert Haas wrote: I think this one could be removed: + if (n 0) + elog(DEBUG1,Absorbing %d fsync requests,n); Right; that one is just there to let you know the patch is working, and how much the background writer does for you per pass, mainly for the purpose of

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: It might be even better to mention that the reason why we couldn't forward the fsync request is that the fsync request queue is full. I'm not sure exactly how to phrase that. I thought about: fsync request queue is full But that seems not to answer

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 7:19 PM, Greg Smith g...@2ndquadrant.com wrote: But if this is generating a lot of log data or adding a lot of overhead, then you have bigger problems anyway: +               elog(DEBUG1, Unable to forward fsync request, executing directly); The argument against

Re: [HACKERS] Count backend self-sync calls

2010-11-14 Thread Robert Haas
On Sun, Nov 14, 2010 at 7:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: It might be even better to mention that the reason why we couldn't forward the fsync request is that the fsync request queue is full. I'm not sure exactly how to phrase that.  I