On Tue, 27 Mar 2007, Magnus Hagander wrote:
Would not at least some of these numbers be better presented through the
stats collector, so they can be easily monitored?
That goes along the line of my way way way away from finished attempt
earlier, perhaps a combination of these two patches?
When
Magnus Hagander wrote:
> Would not at least some of these numbers be better presented through the
> stats collector, so they can be easily monitored?
>
> That goes along the line of my way way way away from finished attempt
> earlier, perhaps a combination of these two patches?
Yes.
Would not at least some of these numbers be better presented through the
stats collector, so they can be easily monitored?
That goes along the line of my way way way away from finished attempt
earlier, perhaps a combination of these two patches?
//Magnus
Bruce Momjian wrote:
> Is this patch rea
Is this patch ready?
---
Greg Smith wrote:
> I have a WIP patch that adds the main detail I have found I need to
> properly tune checkpoint and background writer activity. I think it's
> almost ready to submit (you can se
On Mon, 12 Mar 2007, Tom Lane wrote:
It might also be interesting to know exactly how many buffers were
pinned at the time the scan passed over them. In theory it should be a
small fraction, but maybe it isn't ...
It is; the theory holds for all the tests I tried today. The actual
pinned bu
Greg Smith <[EMAIL PROTECTED]> writes:
> Here's what I get as statistics on the buffer pool after a
> scan when the server is "happy", from a run with 20 clients:
> writes=38.3MB (8.2%) pinned+used=38.3MB (8.2%)
> dirty buffer usage count histogram:
> 0=0.1% 1=0.3% 2=26% 3=17% 4=21% 5+=36%
Inter
On Fri, 9 Mar 2007, Tom Lane wrote:
It strikes me that the patch would be more useful if it produced a
histogram of the observed usage_counts
Don't have something worth releasing yet, but I did code a first rev of
this today. The results are quite instructive and it's well worth looking
at.
On Fri, 9 Mar 2007, Tom Lane wrote:
I'd be interested to know what scale factor and shared_buffers setting
led to the above measurement.
That was just a trivial example with 1 client, scale=10 (~160MB database),
and shared_buffers=24MB. Where things really get interesting with pgbench
is on
On Fri, 9 Mar 2007, Jim Nasby wrote:
I'm wondering if pg_bench is a good test of this stuff. ISTM it's
unrealistically write-heavy, which is going to tend to not only put a
lot of dirty buffers into the pool, but also keep them pinned enough
that you can't write them.
Whether it's "unrealist
Jim Nasby <[EMAIL PROTECTED]> writes:
> On Mar 8, 2007, at 11:51 PM, Greg Smith wrote:
>> almost everything that's dirty is also pinned during pgbench, and
>> the LRU is lucky to find anything it can write as a result
> I'm wondering if pg_bench is a good test of this stuff.
On reflection I thi
On Mar 9, 2007, at 7:57 AM, Greg Smith wrote:
On Fri, 9 Mar 2007, ITAGAKI Takahiro wrote:
"Pinned" means bufHdr->refcount > 0 and you don't distinguish
pinned or recently-used (bufHdr->usage_count > 0) buffers in your
patch.
Thank you, I will revise the terminology used accordingly. I was
On Mar 8, 2007, at 11:51 PM, Greg Smith wrote:
almost everything that's dirty is also pinned during pgbench, and
the LRU is lucky to find anything it can write as a result
I'm wondering if pg_bench is a good test of this stuff. ISTM it's
unrealistically write-heavy, which is going to tend to
On Mar 8, 2007, at 11:51 PM, Greg Smith wrote:
almost everything that's dirty is also pinned during pgbench, and
the LRU is lucky to find anything it can write as a result
I'm wondering if pg_bench is a good test of this stuff. ISTM it's
unrealistically write-heavy, which is going to tend to
On Fri, 9 Mar 2007, ITAGAKI Takahiro wrote:
"Pinned" means bufHdr->refcount > 0 and you don't distinguish pinned or
recently-used (bufHdr->usage_count > 0) buffers in your patch.
Thank you, I will revise the terminology used accordingly. I was using
"pinned" as a shortcut for "will be ignore
Greg Smith <[EMAIL PROTECTED]> wrote:
> > In my understanding, each backend pins two or so buffers at once. So
> > percentage of pinned buffers should be low.
>
> With the pgbench workload, a substantial percentage of the buffer cache
> ends up pinned.
> http://westnet.com/~gsmith/content/pos
Greg Smith <[EMAIL PROTECTED]> writes:
> With the pgbench workload, a substantial percentage of the buffer cache
> ends up pinned.
[ raised eyebrow... ] Prove that. AFAIK it's impossible for the
pgbench queries to pin more than about three or four buffers per backend
concurrently.
On Fri, 9 Mar 2007, ITAGAKI Takahiro wrote:
In my understanding, each backend pins two or so buffers at once. So
percentage of pinned buffers should be low.
With the pgbench workload, a substantial percentage of the buffer cache
ends up pinned. From staring at the buffer cache using
contrib
Greg Smith <[EMAIL PROTECTED]> wrote:
> > Also, my recommended bgwriter_lru_maxpages is "average number of
> > recycled buffers per cycle", that is hardly able to tune manually.
>
> This is completely dependent on what percentage of your buffer cache is
> pinned.
Don't you mean usage_count? In
Jim Nasby <[EMAIL PROTECTED]> wrote:
> > Also, my recommended bgwriter_lru_maxpages is "average number of
> > recycled buffers per cycle", that is hardly able to tune manually.
>
> What do you mean by 'number of recycled buffers per cycle"?
There is the following description in the documentation
On Wed, 7 Mar 2007, ITAGAKI Takahiro wrote:
Also, my recommended bgwriter_lru_maxpages is "average number of
recycled buffers per cycle", that is hardly able to tune manually.
This is completely dependent on what percentage of your buffer cache is
pinned. If your load is something like the s
On Mar 6, 2007, at 10:11 PM, ITAGAKI Takahiro wrote:
I have some results that if we have plenty of time for checkpoints,
bgwriter_all_maxpages is not a so important parameter because it is
adjusted to "shared_buffers / duration of checkpoint".
Also, my recommended bgwriter_lru_maxpages is "averag
Greg Smith <[EMAIL PROTECTED]> wrote:
> After a few months of staring at this data, I've found averages like that
> misleading. The real problem areas correlate with the peak pages written
> at any one checkpoint. Lowering that value is really the end-game for
> optimizing the background writ
On Tue, 6 Mar 2007, Jim Nasby wrote:
The flipside is that it's much easier to machine-parse a table rather
than trying to scrape the logs.
Now you might realize why I've been so vocal on the SQL log export
implementation details.
And I don't think we'll generally care about each individual
On Mar 5, 2007, at 8:34 PM, Greg Smith wrote:
On Thu, 22 Feb 2007, Jim C. Nasby wrote:
It would also be extremely useful to make checkpoint stats visible
somewhere in the database (presumably via the existing stats
mechanism)... I'm thinking just tracking how many pages had to be
flushed d
On Wed, 21 Feb 2007, Robert Treat wrote:
My impression of this is that DBA's would typically want to run this for a
short period of time to get thier systems tuned and then it pretty much
becomes chatter. Can you come up with an idea of what information DBA's need
to know?
I am structing the
On Thu, 22 Feb 2007, Jim C. Nasby wrote:
It would also be extremely useful to make checkpoint stats visible
somewhere in the database (presumably via the existing stats
mechanism)... I'm thinking just tracking how many pages had to be
flushed during a checkpoint would be a good start.
I'm in
On Mon, Feb 19, 2007 at 10:59:38PM -0500, Greg Smith wrote:
> I have a WIP patch that adds the main detail I have found I need to
> properly tune checkpoint and background writer activity. I think it's
> almost ready to submit (you can see the current patch against 8.2 at
> http://www.westnet.c
On Monday 19 February 2007 22:59, Greg Smith wrote:
> I have a WIP patch that adds the main detail I have found I need to
> properly tune checkpoint and background writer activity. I think it's
> almost ready to submit (you can see the current patch against 8.2 at
> http://www.westnet.com/~gsmith/
I have a WIP patch that adds the main detail I have found I need to
properly tune checkpoint and background writer activity. I think it's
almost ready to submit (you can see the current patch against 8.2 at
http://www.westnet.com/~gsmith/content/postgresql/patch-checkpoint.txt )
after making i
29 matches
Mail list logo
