On Fri, Jan 17, 2014 at 8:20 AM, Magnus Hagander wrote:
>> Robert Haas reported that setting it to 32MB can yield a considerable
>> performance benefit:
>>
>> http://www.postgresql.org/message-id/CA+TgmobgMv_aaakLoasBt=5iYfi=kdcOUz0X9TdYe0c2SZ=2...@mail.gmail.com
>
> In that case, sholdn't the aut
On 17 January 2014 13:20, Magnus Hagander wrote:
> On Fri, Jan 17, 2014 at 2:07 PM, Thom Brown wrote:
>>
>> On 17 January 2014 13:01, Magnus Hagander wrote:
>> > Is there any real use-case for not setting wal_buffers to -1 these days?
>> >
>> > Or should we just remove it and use the -1 behaviou
On Fri, Jan 17, 2014 at 2:07 PM, Thom Brown wrote:
> On 17 January 2014 13:01, Magnus Hagander wrote:
> > Is there any real use-case for not setting wal_buffers to -1 these days?
> >
> > Or should we just remove it and use the -1 behaviour at all times?
> >
> > IIRC we discussed not keeping it a
Hi,
On 2014-01-17 14:01:56 +0100, Magnus Hagander wrote:
> Is there any real use-case for not setting wal_buffers to -1 these days?
>
> Or should we just remove it and use the -1 behaviour at all times?
I have seen improvements by setting it larger than the max -1 one
value. Also, for some workl
On 17 January 2014 13:01, Magnus Hagander wrote:
> Is there any real use-case for not setting wal_buffers to -1 these days?
>
> Or should we just remove it and use the -1 behaviour at all times?
>
> IIRC we discussed not keeping it at all when the autotune behavior was
> introduced, but said we wa
Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we just remove it and use the -1 behaviour at all times?
IIRC we discussed not keeping it at all when the autotune behavior was
introduced, but said we wanted to keep it "just in case". If we're not
ready to remove
On Thursday, August 30, 2012 7:14 PM Robert Haas wrote:
On Wed, Aug 29, 2012 at 10:25 PM, Peter Geoghegan
wrote:
> On 19 February 2012 05:24, Robert Haas wrote:
>>> I have attached tps scatterplots. The obvious conclusion appears to
>>> be that, with only 16MB of wal_buffers, the buffer "wraps a
On Wed, Aug 29, 2012 at 10:25 PM, Peter Geoghegan wrote:
> On 19 February 2012 05:24, Robert Haas wrote:
>> I have attached tps scatterplots. The obvious conclusion appears to
>> be that, with only 16MB of wal_buffers, the buffer "wraps around" with
>> some regularity: we can't insert more WAL b
On 19 February 2012 05:24, Robert Haas wrote:
> I have attached tps scatterplots. The obvious conclusion appears to
> be that, with only 16MB of wal_buffers, the buffer "wraps around" with
> some regularity: we can't insert more WAL because the buffer we need
> to use still contains WAL that hasn
On Tuesday, August 28, 2012 9:33 PM Bruce Momjian wrote:
On Tue, Aug 28, 2012 at 09:40:33AM +0530, Amit Kapila wrote:
> From: pgsql-hackers-ow...@postgresql.org
> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
>
>>> Added to TODO:
>
>>> Allow reporting of stalls due to
On Tue, Aug 28, 2012 at 09:40:33AM +0530, Amit Kapila wrote:
> From: pgsql-hackers-ow...@postgresql.org
> [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
>
> > Added to TODO:
>
> > Allow reporting of stalls due to wal_buffer wrap-around
>
> >
> http://archives.po
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
> Added to TODO:
> Allow reporting of stalls due to wal_buffer wrap-around
>
http://archives.postgresql.org/pgsql-hackers/2012-02/msg00826.php
Isn't this indicates that
Added to TODO:
Allow reporting of stalls due to wal_buffer wrap-around
http://archives.postgresql.org/pgsql-hackers/2012-02/msg00826.php
---
On Sun, Feb 19, 2012 at 12:24:12AM -0500, Robert Haa
On Wed, Mar 14, 2012 at 3:29 PM, Jeff Janes wrote:
> I think my analysis is pretty much a re-wording of yours, but I'd
> emphasize that getting the WALWriteLock is bad not just because they
> fight over the lock, but because someone else (probably background wal
> writer) is camping out on the loc
On Tue, Mar 13, 2012 at 11:18 PM, Robert Haas wrote:
> On Tue, Mar 13, 2012 at 6:44 PM, Josh Berkus wrote:
>>> That's a speedup of nearly a factor of two, so clearly fsync-related
>>> stalls are a big problem here, even with wal_buffers cranked up
>>> through the ceiling.
>>
>> H. Do you ha
On Mon, Mar 12, 2012 at 7:16 AM, Robert Haas wrote:
> On Sun, Mar 11, 2012 at 11:51 PM, Fujii Masao wrote:
>> On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas wrote:
>>> I've finally been able to run some more tests of the effect of
>>> adjusting wal_buffers to values higher than 16MB. I ran the t
On Tue, Mar 13, 2012 at 6:44 PM, Josh Berkus wrote:
>> That's a speedup of nearly a factor of two, so clearly fsync-related
>> stalls are a big problem here, even with wal_buffers cranked up
>> through the ceiling.
>
> H. Do you have any ability to test on XFS?
It seems I do.
XFS, with fsy
On Tue, Mar 13, 2012 at 10:02 PM, Fujii Masao wrote:
> On Wed, Mar 14, 2012 at 7:20 AM, Robert Haas wrote:
>> On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas wrote:
>>> On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
m
On Wed, Mar 14, 2012 at 7:20 AM, Robert Haas wrote:
> On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas wrote:
>> On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
>>> Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
>>> machines) with fsync=off (as well as synchronous_commit=off s
> That's a speedup of nearly a factor of two, so clearly fsync-related
> stalls are a big problem here, even with wal_buffers cranked up
> through the ceiling.
H. Do you have any ability to test on XFS?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hacker
On Tue, Mar 13, 2012 at 4:55 AM, Andres Freund wrote:
> On Tuesday, March 13, 2012 03:26:34 AM Robert Haas wrote:
>> Meanwhile, here are some TPS graphs at 16MB and 32MB on the IBM POWER7
>> machine. 32 clients, 1800 seconds, scale factor 300, synchronous
>> commit off.
> That graph makes me crin
On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas wrote:
> On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
>> Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
>> machines) with fsync=off (as well as synchronous_commit=off still)
>> might help clarify things.
>
> I reran the 32-cli
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
> Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
> machines) with fsync=off (as well as synchronous_commit=off still)
> might help clarify things.
I reran the 32-client benchmark on the IBM machine with fsync=off and got this:
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
> Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
> machines) with fsync=off (as well as synchronous_commit=off still)
> might help clarify things.
> If it increases the TPS of Nate@16MB, but doesn't change the other 3
> situati
On Tuesday, March 13, 2012 03:26:34 AM Robert Haas wrote:
> Meanwhile, here are some TPS graphs at 16MB and 32MB on the IBM POWER7
> machine. 32 clients, 1800 seconds, scale factor 300, synchronous
> commit off.
That graph makes me cringe because its pretty representative of what I have
seen in p
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes wrote:
>>> Do you think the difference is in the CPU architecture, or the
>>> IO subsystem?
>>
>> That is an excellent question. I tried looking at vmstat output, but
>> a funny thing kept happening: periodically, the iowait column would
>> show a gigan
On Mon, Mar 12, 2012 at 10:55 AM, Robert Haas wrote:
> On Mon, Mar 12, 2012 at 12:32 PM, Jeff Janes wrote:
>> On Nate Boley's machine, the difference was ~100% increase rather than
>> ~10%.
>
> Oh, right. I had forgotten how dramatic the changes were in those
> test runs. I guess I should be ha
On Mon, Mar 12, 2012 at 12:32 PM, Jeff Janes wrote:
> On Nate Boley's machine, the difference was ~100% increase rather than
> ~10%.
Oh, right. I had forgotten how dramatic the changes were in those
test runs. I guess I should be happy that the absolute numbers on
this machine were as high as t
On Sat, Mar 10, 2012 at 7:55 PM, Robert Haas wrote:
> I've finally been able to run some more tests of the effect of
> adjusting wal_buffers to values higher than 16MB. I ran the test on
> the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
> configuration settings:
>
> shared_buf
On Sun, Mar 11, 2012 at 11:51 PM, Fujii Masao wrote:
> On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas wrote:
>> I've finally been able to run some more tests of the effect of
>> adjusting wal_buffers to values higher than 16MB. I ran the test on
>> the 16 core (x 4 hw threads/core) IBM POWER7 mac
On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas wrote:
> I've finally been able to run some more tests of the effect of
> adjusting wal_buffers to values higher than 16MB. I ran the test on
> the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
> configuration settings:
>
> shared_bu
I've finally been able to run some more tests of the effect of
adjusting wal_buffers to values higher than 16MB. I ran the test on
the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
configuration settings:
shared_buffers = 8GB
maintenance_work_mem = 1GB
synchronous_commit = off
c
On Mon, Feb 20, 2012 at 3:59 AM, Simon Riggs wrote:
>> There is no existing statistics view suitable to include such information.
>> What about defining pg_stat_xlog or something?
>
> Perhaps pg_stat_perf so we don't need to find a new home every time.
>
> Thinking about it, I think renaming pg_st
On Mon, Feb 20, 2012 at 4:10 AM, Fujii Masao wrote:
> On Mon, Feb 20, 2012 at 3:08 AM, Robert Haas wrote:
>> On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
>> wrote:
>>> On 19-02-2012 02:24, Robert Haas wrote:
I have attached tps scatterplots. The obvious conclusion appears to
On 02/19/2012 12:24 AM, Robert Haas wrote:
I think we might want to consider
adjusting our auto-tuning formula for wal_buffers to allow for a
higher cap, although this is obviously not enough data to draw any
firm conclusions.
That's an easy enough idea to throw into my testing queue. The 16MB
On Mon, Feb 20, 2012 at 3:08 AM, Robert Haas wrote:
> On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
> wrote:
>> On 19-02-2012 02:24, Robert Haas wrote:
>>> I have attached tps scatterplots. The obvious conclusion appears to
>>> be that, with only 16MB of wal_buffers, the buffer "wra
On Sun, Feb 19, 2012 at 6:33 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
>> wrote:
>>> Isn't it useful to print some messages on the log when we have "wrap
>>> around"?
>>> In this case, we have an idea that wal_buffers needs to be inc
Robert Haas writes:
> On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
> wrote:
>> Isn't it useful to print some messages on the log when we have "wrap around"?
>> In this case, we have an idea that wal_buffers needs to be increased.
> I was thinking about that. I think that what migh
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
wrote:
> On 19-02-2012 02:24, Robert Haas wrote:
>> I have attached tps scatterplots. The obvious conclusion appears to
>> be that, with only 16MB of wal_buffers, the buffer "wraps around" with
>> some regularity
>>
> Isn't it useful to p
On 19-02-2012 02:24, Robert Haas wrote:
> I have attached tps scatterplots. The obvious conclusion appears to
> be that, with only 16MB of wal_buffers, the buffer "wraps around" with
> some regularity
>
Isn't it useful to print some messages on the log when we have "wrap around"?
In this case, we
Just for kicks, I ran two 30-minute pgbench tests at scale factor 300
tonight on Nate Boley's machine, with -n -l -c 32 -j 32. The
configurations were identical, except that on one of them, I set
wal_buffers=64MB. It seemed to make quite a lot of difference:
wal_buffers not set (thus, 16MB):
tps
On Mon, Apr 4, 2011 at 4:57 PM, Tom Lane wrote:
> Robert Haas writes:
>> OK. Please comment the crap out of whatever you do, or maybe even add
>> a README. This stuff is just a bit arcane, and guideposts help a lot.
>
> We already have a README for that ;-). PFA, a patch to
> src/backend/utils
Robert Haas writes:
> OK. Please comment the crap out of whatever you do, or maybe even add
> a README. This stuff is just a bit arcane, and guideposts help a lot.
We already have a README for that ;-). PFA, a patch to
src/backend/utils/misc/README describing the proposed revised API.
If nobo
On Mon, Apr 4, 2011 at 3:14 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane wrote:
>>> Another variant would be to allow the check_hook to pass back a separate
>>> "void *" value that could be passed on to the assign_hook, containing
>>> any necessary derive
Robert Haas writes:
> On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane wrote:
>> Another variant would be to allow the check_hook to pass back a separate
>> "void *" value that could be passed on to the assign_hook, containing
>> any necessary derived data. This is logically a bit cleaner, and would
>>
On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane wrote:
>>> Given these rules, a check_hook and assign_hook could cooperate to store
>>> additional data in what guc.c thinks is just a pointer to a string
>>> value, ie, there can be
Robert Haas writes:
> On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane wrote:
>> Given these rules, a check_hook and assign_hook could cooperate to store
>> additional data in what guc.c thinks is just a pointer to a string
>> value, ie, there can be more data after the terminating \0. The
>> assign_hoo
On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane wrote:
> I wrote:
>> IMO the real problem is essentially that GUC assign hooks have two
>> functions, checking and canonicalization of the value-to-be-stored
>> versus executing secondary actions when an assignment is made; and
>> there's no way to get at j
I wrote:
> IMO the real problem is essentially that GUC assign hooks have two
> functions, checking and canonicalization of the value-to-be-stored
> versus executing secondary actions when an assignment is made; and
> there's no way to get at just the first one. So we cannot canonicalize
> the val
Robert Haas wrote:
> On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane wrote:
>> It would probably take less than a day to flesh out this idea and
>> make it happen, but it does seem like a rather large change for
>> late alpha.
> what we're trying to avoid is committing new stuff that may require
> a
Robert Haas writes:
> On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane wrote:
>> IMO the real problem is essentially that GUC assign hooks have two
>> functions, checking and canonicalization of the value-to-be-stored
>> versus executing secondary actions when an assignment is made; and
>> there's no way
On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane wrote:
> IMO the real problem is essentially that GUC assign hooks have two
> functions, checking and canonicalization of the value-to-be-stored
> versus executing secondary actions when an assignment is made; and
> there's no way to get at just the first o
I wrote:
> Robert Haas writes:
>> I had intended to commit Greg's patch with a show hook and an assign
>> hook and the calculated value stored separately from the GUC variable,
>> which I believe would avoid this is a problem, but Tom thought this
>> way was better. Unfortunately, my knowledge of
Robert Haas writes:
> On Thu, Mar 31, 2011 at 8:38 AM, Bernd Helmle wrote:
>> This might be nitpicking (or i'm currently missing something), but i
>> recognized that setting wal_buffers = -1 always triggers the following on
>> reload, even if nothing to wal_buffers had changed:
>>
>> $ pg_ctl re
On Thu, Mar 31, 2011 at 8:38 AM, Bernd Helmle wrote:
> This might be nitpicking (or i'm currently missing something), but i
> recognized that setting wal_buffers = -1 always triggers the following on
> reload, even if nothing to wal_buffers had changed:
>
> $ pg_ctl reload
> LOG: received SIGHUP,
This might be nitpicking (or i'm currently missing something), but i recognized
that setting wal_buffers = -1 always triggers the following on reload, even if
nothing to wal_buffers had changed:
$ pg_ctl reload
LOG: received SIGHUP, reloading configuration files
LOG: parameter "wal_buffers" c
56 matches
Mail list logo
