On Fri, Sep 1, 2017 at 9:17 PM, Robert Haas wrote:
> On Fri, Sep 1, 2017 at 10:03 AM, Dilip Kumar wrote:
>>> Sure will do so. In the meantime, I have rebased the patch.
>>
>> I have repeated some of the tests we have performed earlier.
>
Thanks for repeating the performance tests.
> OK, these
On Fri, Sep 1, 2017 at 10:03 AM, Dilip Kumar wrote:
>> Sure will do so. In the meantime, I have rebased the patch.
>
> I have repeated some of the tests we have performed earlier.
OK, these tests seem to show that this is still working. Committed,
again. Let's hope this attempt goes better tha
On Wed, Aug 30, 2017 at 12:54 PM, Amit Kapila wrote:
>
> That would have been better. In any case, will do the tests on some
> higher end machine and will share the results.
>
>> Given that we've changed the approach here somewhat, I think we need
>> to validate that we're still seeing a substanti
On Wed, Aug 30, 2017 at 2:43 AM, Robert Haas wrote:
> On Tue, Jul 4, 2017 at 12:33 AM, Amit Kapila wrote:
>> I have updated the patch to support wait events and moved it to upcoming CF.
>
> This patch doesn't apply any more, but I made it apply with a hammer
> and then did a little benchmarking (
On Tue, Jul 4, 2017 at 12:33 AM, Amit Kapila wrote:
> I have updated the patch to support wait events and moved it to upcoming CF.
This patch doesn't apply any more, but I made it apply with a hammer
and then did a little benchmarking (scylla, EDB server, Intel Xeon
E5-2695 v3 @ 2.30GHz, 2 socket
On Mon, Jul 3, 2017 at 6:15 PM, Amit Kapila wrote:
> On Thu, Mar 23, 2017 at 1:18 PM, Ashutosh Sharma
> wrote:
>>
>> Conclusion:
>> As seen from the test results mentioned above, there is some performance
>> improvement with 3 SP(s), with 5 SP(s) the results with patch is slightly
>> better than
On Thu, Mar 23, 2017 at 1:18 PM, Ashutosh Sharma
wrote:
>
> *Conclusion:*
> As seen from the test results mentioned above, there is some performance
> improvement with 3 SP(s), with 5 SP(s) the results with patch is slightly
> better than HEAD, with 7 and 10 SP(s) we do see regression with patch.
On Thu, Mar 9, 2017 at 5:49 PM, Robert Haas wrote:
> However, I just realized that in
> both this case and in the case of group XID clearing, we weren't
> advertising a wait event for the PGSemaphoreLock calls that are part
> of the group locking machinery. I think we should fix that, because a
>
Hi All,
I have tried to test 'group_update_clog_v11.1.patch' shared upthread by
Amit on a high end machine. I have tested the patch with various savepoints
in my test script. The machine details along with test scripts and the test
results are shown below,
Machine details:
24 sockets
On Mon, Mar 20, 2017 at 8:27 AM, Robert Haas wrote:
> On Fri, Mar 17, 2017 at 2:30 AM, Amit Kapila wrote:
>>> I was wondering about doing an explicit test: if the XID being
>>> committed matches the one in the PGPROC, and nsubxids matches, and the
>>> actual list of XIDs matches, then apply the o
On Fri, Mar 17, 2017 at 2:30 AM, Amit Kapila wrote:
>> I was wondering about doing an explicit test: if the XID being
>> committed matches the one in the PGPROC, and nsubxids matches, and the
>> actual list of XIDs matches, then apply the optimization. That could
>> replace the logic that you've
On Sun, Mar 12, 2017 at 8:11 AM, Robert Haas wrote:
> On Fri, Mar 10, 2017 at 7:39 PM, Amit Kapila wrote:
>> I agree that more analysis can help us to decide if we can use subxids
>> from PGPROC and if so under what conditions. Have you considered the
>> another patch I have posted to fix the is
On Fri, Mar 10, 2017 at 7:39 PM, Amit Kapila wrote:
> I agree that more analysis can help us to decide if we can use subxids
> from PGPROC and if so under what conditions. Have you considered the
> another patch I have posted to fix the issue which is to do this
> optimization only when subxids a
On Sat, Mar 11, 2017 at 2:10 AM, Robert Haas wrote:
> On Fri, Mar 10, 2017 at 6:25 AM, Amit Kapila wrote:
>> On Fri, Mar 10, 2017 at 11:51 AM, Tom Lane wrote:
>>> Amit Kapila writes:
Just to let you know that I think I have figured out the reason of
failure. If we run the regressions
Robert Haas wrote:
> The smoking gun was in 009_twophase_slave.log:
>
> TRAP: FailedAssertion("!(nsubxids == MyPgXact->nxids)", File:
> "clog.c", Line: 288)
>
> ...and then the node shuts down, which is why this hangs forever.
> (Also... what's up with it hanging forever instead of timing out or
On Fri, Mar 10, 2017 at 3:40 PM, Robert Haas wrote:
> Finally, I had an unexplained hang during the TAP tests while testing
> out your fix patch. I haven't been able to reproduce that so it
> might've just been an artifact of something stupid I did, or of some
> unrelated bug, but I think it's be
On Fri, Mar 10, 2017 at 6:25 AM, Amit Kapila wrote:
> On Fri, Mar 10, 2017 at 11:51 AM, Tom Lane wrote:
>> Amit Kapila writes:
>>> Just to let you know that I think I have figured out the reason of
>>> failure. If we run the regressions with attached patch, it will make
>>> the regression tests
On Fri, Mar 10, 2017 at 11:51 AM, Tom Lane wrote:
> Amit Kapila writes:
>> Just to let you know that I think I have figured out the reason of
>> failure. If we run the regressions with attached patch, it will make
>> the regression tests fail consistently in same way. The patch just
>> makes al
On Fri, Mar 10, 2017 at 11:43 AM, Amit Kapila wrote:
> On Fri, Mar 10, 2017 at 10:51 AM, Tom Lane wrote:
>>
>> Also, I see clam reported in green just now, so it's not 100%
>> reproducible :-(
>>
>
> Just to let you know that I think I have figured out the reason of
> failure. If we run the regr
Amit Kapila writes:
> Just to let you know that I think I have figured out the reason of
> failure. If we run the regressions with attached patch, it will make
> the regression tests fail consistently in same way. The patch just
> makes all transaction status updates to go via group clog update
On Fri, Mar 10, 2017 at 10:51 AM, Tom Lane wrote:
> Robert Haas writes:
>> On Thu, Mar 9, 2017 at 9:17 PM, Tom Lane wrote:
>>> Buildfarm thinks eight wasn't enough.
>>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=clam&dt=2017-03-10%2002%3A00%3A01
>
>> At first I was confused how you
Robert Haas writes:
> On Thu, Mar 9, 2017 at 9:17 PM, Tom Lane wrote:
>> Buildfarm thinks eight wasn't enough.
>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=clam&dt=2017-03-10%2002%3A00%3A01
> At first I was confused how you knew that this was the fault of this
> patch, but this see
On Thu, Mar 9, 2017 at 9:17 PM, Tom Lane wrote:
> Robert Haas writes:
>> I think eight is enough. Committed with some cosmetic changes.
>
> Buildfarm thinks eight wasn't enough.
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=clam&dt=2017-03-10%2002%3A00%3A01
At first I was confused
On Fri, Mar 10, 2017 at 7:47 AM, Tom Lane wrote:
> Robert Haas writes:
>> I think eight is enough. Committed with some cosmetic changes.
>
> Buildfarm thinks eight wasn't enough.
>
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=clam&dt=2017-03-10%2002%3A00%3A01
>
Will look into this,
Robert Haas writes:
> I think eight is enough. Committed with some cosmetic changes.
Buildfarm thinks eight wasn't enough.
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=clam&dt=2017-03-10%2002%3A00%3A01
regards, tom lane
--
Sent via pgsql-hackers mailing li
On Tue, Jan 31, 2017 at 11:35 PM, Michael Paquier
wrote:
>> Thanks for the review.
>
> Moved to CF 2017-03, the 8th commit fest of this patch.
I think eight is enough. Committed with some cosmetic changes.
I think the turning point for this somewhat-troubled patch was when we
realized that, whi
On Tue, Jan 17, 2017 at 9:18 PM, Amit Kapila wrote:
> On Tue, Jan 17, 2017 at 11:39 AM, Dilip Kumar wrote:
>> On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote:
>>> I have reviewed the latest patch and I don't have any more comments.
>>> So if there is no objection from other reviewers I can m
On Tue, Jan 17, 2017 at 11:39 AM, Dilip Kumar wrote:
> On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote:
>> I have reviewed the latest patch and I don't have any more comments.
>> So if there is no objection from other reviewers I can move it to
>> "Ready For Committer"?
>
> Seeing no objectio
On Wed, Jan 11, 2017 at 10:55 AM, Dilip Kumar wrote:
> I have reviewed the latest patch and I don't have any more comments.
> So if there is no objection from other reviewers I can move it to
> "Ready For Committer"?
Seeing no objections, I have moved it to Ready For Committer.
--
Regards,
Dil
On Sat, Dec 31, 2016 at 9:01 AM, Amit Kapila wrote:
> Agreed and changed accordingly.
>
>> 2. It seems that we have missed one unlock in case of absorbed
>> wakeups. You have initialised extraWaits with -1 and if there is one
>> extra wake up then extraWaits will become 0 (it means we have made on
On Thu, Dec 29, 2016 at 10:41 AM, Dilip Kumar wrote:
>
> I have done one more pass of the review today. I have few comments.
>
> + if (nextidx != INVALID_PGPROCNO)
> + {
> + /* Sleep until the leader updates our XID status. */
> + for (;;)
> + {
> + /* acts as a read barrier */
> + PGSemaphoreLock
On Fri, Dec 23, 2016 at 8:28 AM, Amit Kapila wrote:
> The results look positive. Do you think we can conclude based on all
> the tests you and Dilip have done, that we can move forward with this
> patch (in particular group-update) or do you still want to do more
> tests? I am aware that in one
On 12/23/2016 03:58 AM, Amit Kapila wrote:
On Thu, Dec 22, 2016 at 6:59 PM, Tomas Vondra
wrote:
Hi,
But as discussed with Amit in Tokyo at pgconf.asia, I got access to a
Power8e machine (IBM 8247-22L to be precise). It's a much smaller machine
compared to the x86 one, though - it only has 24 c
On Thu, Dec 22, 2016 at 6:59 PM, Tomas Vondra
wrote:
> Hi,
>
> But as discussed with Amit in Tokyo at pgconf.asia, I got access to a
> Power8e machine (IBM 8247-22L to be precise). It's a much smaller machine
> compared to the x86 one, though - it only has 24 cores in 2 sockets, 128GB
> of RAM and
Hi,
The attached results show that:
(a) master shows the same zig-zag behavior - No idea why this wasn't
observed on the previous runs.
(b) group_update actually seems to improve the situation, because the
performance keeps stable up to 72 clients, while on master the
fluctuation starts way ea
On Mon, Dec 5, 2016 at 1:14 PM, Amit Kapila wrote:
> On Mon, Dec 5, 2016 at 6:00 AM, Haribabu Kommi
> wrote:
>
> No, that is not true. You have quoted the wrong message, that
> discussion was about WALWriteLock contention not about the patch being
> discussed in this thread. I have posted the
On Mon, Dec 5, 2016 at 6:00 AM, Haribabu Kommi wrote:
>
>
> On Fri, Nov 4, 2016 at 8:20 PM, Amit Kapila wrote:
>>
>> On Thu, Nov 3, 2016 at 8:38 PM, Robert Haas wrote:
>> > On Tue, Nov 1, 2016 at 11:31 PM, Tomas Vondra
>> >> The difference is that both the fast-path locks and msgNumLock went
>>
On Fri, Nov 4, 2016 at 8:20 PM, Amit Kapila wrote:
> On Thu, Nov 3, 2016 at 8:38 PM, Robert Haas wrote:
> > On Tue, Nov 1, 2016 at 11:31 PM, Tomas Vondra
> >> The difference is that both the fast-path locks and msgNumLock went into
> >> 9.2, so that end users probably never saw that regression.
On Thu, Nov 3, 2016 at 8:38 PM, Robert Haas wrote:
> On Tue, Nov 1, 2016 at 11:31 PM, Tomas Vondra
>> The difference is that both the fast-path locks and msgNumLock went into
>> 9.2, so that end users probably never saw that regression. But we don't know
>> if that happens for clog and WAL.
>>
>>
On Tue, Nov 1, 2016 at 11:31 PM, Tomas Vondra
wrote:
> I don't think I've suggested not committing any of the clog patches (or
> other patches in general) because shifting the contention somewhere else
> might cause regressions. At the end of the last CF I've however stated that
> we need to bette
On 11/02/2016 05:52 PM, Amit Kapila wrote:
On Wed, Nov 2, 2016 at 9:01 AM, Tomas Vondra
wrote:
On 11/01/2016 08:13 PM, Robert Haas wrote:
On Mon, Oct 31, 2016 at 5:48 PM, Tomas Vondra
wrote:
The one remaining thing is the strange zig-zag behavior, but that might
easily be a due to schedu
On 11/02/2016 05:52 PM, Amit Kapila wrote:
On Wed, Nov 2, 2016 at 9:01 AM, Tomas Vondra
wrote:
On 11/01/2016 08:13 PM, Robert Haas wrote:
On Mon, Oct 31, 2016 at 5:48 PM, Tomas Vondra
wrote:
The one remaining thing is the strange zig-zag behavior, but that might
easily be a due to schedu
On Wed, Nov 2, 2016 at 9:01 AM, Tomas Vondra
wrote:
> On 11/01/2016 08:13 PM, Robert Haas wrote:
>>
>> On Mon, Oct 31, 2016 at 5:48 PM, Tomas Vondra
>> wrote:
>>>
>
> The one remaining thing is the strange zig-zag behavior, but that might
> easily be a due to scheduling in kernel, or something el
On 11/01/2016 08:13 PM, Robert Haas wrote:
On Mon, Oct 31, 2016 at 5:48 PM, Tomas Vondra
wrote:
Honestly, I have no idea what to think about this ...
I think a lot of the details here depend on OS scheduler behavior.
For example, here's one of the first scalability graphs I ever did:
http://
On Mon, Oct 31, 2016 at 5:48 PM, Tomas Vondra
wrote:
> Honestly, I have no idea what to think about this ...
I think a lot of the details here depend on OS scheduler behavior.
For example, here's one of the first scalability graphs I ever did:
http://rhaas.blogspot.com/2011/09/scalability-in-gra
On 10/31/2016 02:24 PM, Tomas Vondra wrote:
On 10/31/2016 05:01 AM, Jim Nasby wrote:
On 10/30/16 1:32 PM, Tomas Vondra wrote:
Now, maybe this has nothing to do with PostgreSQL itself, but maybe it's
some sort of CPU / OS scheduling artifact. For example, the system has
36 physical cores, 72 vi
On 10/31/2016 08:43 PM, Amit Kapila wrote:
On Mon, Oct 31, 2016 at 7:58 PM, Tomas Vondra
wrote:
On 10/31/2016 02:51 PM, Amit Kapila wrote:
And moreover, this setup (single device for the whole cluster) is very
common, we can't just neglect it.
But my main point here really is that the trade-of
On Mon, Oct 31, 2016 at 7:58 PM, Tomas Vondra
wrote:
> On 10/31/2016 02:51 PM, Amit Kapila wrote:
> And moreover, this setup (single device for the whole cluster) is very
> common, we can't just neglect it.
>
> But my main point here really is that the trade-off in those cases may not
> be really
On 10/31/2016 02:51 PM, Amit Kapila wrote:
On Mon, Oct 31, 2016 at 12:02 AM, Tomas Vondra
wrote:
Hi,
On 10/27/2016 01:44 PM, Amit Kapila wrote:
I've read that analysis, but I'm not sure I see how it explains the "zig
zag" behavior. I do understand that shifting the contention to some other
(a
On Mon, Oct 31, 2016 at 7:02 PM, Tomas Vondra
wrote:
>
> The remaining benchmark with 512 clog buffers completed, and the impact
> roughly matches Dilip's benchmark - that is, increasing the number of clog
> buffers eliminates all positive impact of the patches observed on 128
> buffers. Compare t
On Mon, Oct 31, 2016 at 12:02 AM, Tomas Vondra
wrote:
> Hi,
>
> On 10/27/2016 01:44 PM, Amit Kapila wrote:
>
> I've read that analysis, but I'm not sure I see how it explains the "zig
> zag" behavior. I do understand that shifting the contention to some other
> (already busy) lock may negatively i
On 10/30/2016 07:32 PM, Tomas Vondra wrote:
Hi,
On 10/27/2016 01:44 PM, Amit Kapila wrote:
On Thu, Oct 27, 2016 at 4:15 AM, Tomas Vondra
wrote:
FWIW I plan to run the same test with logged tables - if it shows
similar
regression, I'll be much more worried, because that's a fairly typical
sce
On 10/31/2016 05:01 AM, Jim Nasby wrote:
On 10/30/16 1:32 PM, Tomas Vondra wrote:
Now, maybe this has nothing to do with PostgreSQL itself, but maybe it's
some sort of CPU / OS scheduling artifact. For example, the system has
36 physical cores, 72 virtual ones (thanks to HT). I find it strange
On 10/30/16 1:32 PM, Tomas Vondra wrote:
Now, maybe this has nothing to do with PostgreSQL itself, but maybe it's
some sort of CPU / OS scheduling artifact. For example, the system has
36 physical cores, 72 virtual ones (thanks to HT). I find it strange
that the "good" client counts are always m
Hi,
On 10/27/2016 01:44 PM, Amit Kapila wrote:
On Thu, Oct 27, 2016 at 4:15 AM, Tomas Vondra
wrote:
FWIW I plan to run the same test with logged tables - if it shows similar
regression, I'll be much more worried, because that's a fairly typical
scenario (logged tables, data set > shared buffe
On Thu, Oct 27, 2016 at 5:14 PM, Amit Kapila wrote:
>>> Thanks Tomas and Dilip for doing detailed performance tests for this
>>> patch. I would like to summarise the performance testing results.
>>>
>>> 1. With update intensive workload, we are seeing gains from 23%~192%
>>> at client count >=64
On Thu, Oct 27, 2016 at 4:15 AM, Tomas Vondra
wrote:
> On 10/25/2016 06:10 AM, Amit Kapila wrote:
>>
>> On Mon, Oct 24, 2016 at 2:48 PM, Dilip Kumar
>> wrote:
>>>
>>> On Fri, Oct 21, 2016 at 7:57 AM, Dilip Kumar
>>> wrote:
On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra
wrote:
On 10/25/2016 06:10 AM, Amit Kapila wrote:
On Mon, Oct 24, 2016 at 2:48 PM, Dilip Kumar wrote:
On Fri, Oct 21, 2016 at 7:57 AM, Dilip Kumar wrote:
On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra
wrote:
In the results you've posted on 10/12, you've mentioned a regression with 32
clients, wher
On Mon, Oct 24, 2016 at 2:48 PM, Dilip Kumar wrote:
> On Fri, Oct 21, 2016 at 7:57 AM, Dilip Kumar wrote:
>> On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra
>> wrote:
>>
>>> In the results you've posted on 10/12, you've mentioned a regression with 32
>>> clients, where you got 52k tps on master bu
On Fri, Oct 21, 2016 at 7:57 AM, Dilip Kumar wrote:
> On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra
> wrote:
>
>> In the results you've posted on 10/12, you've mentioned a regression with 32
>> clients, where you got 52k tps on master but only 48k tps with the patch (so
>> ~10% difference). I hav
On Fri, Oct 21, 2016 at 1:07 PM, Tomas Vondra
wrote:
> On 10/21/2016 08:13 AM, Amit Kapila wrote:
>>
>> On Fri, Oct 21, 2016 at 6:31 AM, Robert Haas
>> wrote:
>>>
>>> On Thu, Oct 20, 2016 at 4:04 PM, Tomas Vondra
>>> wrote:
>
> I then started a run at 96 clients which I accidentally kill
On 10/21/2016 08:13 AM, Amit Kapila wrote:
On Fri, Oct 21, 2016 at 6:31 AM, Robert Haas wrote:
On Thu, Oct 20, 2016 at 4:04 PM, Tomas Vondra
wrote:
I then started a run at 96 clients which I accidentally killed shortly
before it was scheduled to finish, but the results are not much
different;
On Fri, Oct 21, 2016 at 6:31 AM, Robert Haas wrote:
> On Thu, Oct 20, 2016 at 4:04 PM, Tomas Vondra
> wrote:
>>> I then started a run at 96 clients which I accidentally killed shortly
>>> before it was scheduled to finish, but the results are not much
>>> different; there is no hint of the runawa
On Thu, Oct 20, 2016 at 9:15 PM, Robert Haas wrote:
> So here's my theory. The whole reason why Tomas is having difficulty
> seeing any big effect from these patches is because he's testing on
> x86. When Dilip tests on x86, he doesn't see a big effect either,
> regardless of workload. But when
On Thu, Oct 20, 2016 at 9:03 PM, Tomas Vondra
wrote:
> In the results you've posted on 10/12, you've mentioned a regression with 32
> clients, where you got 52k tps on master but only 48k tps with the patch (so
> ~10% difference). I have no idea what scale was used for those tests,
That test was
On Thu, Oct 20, 2016 at 4:04 PM, Tomas Vondra
wrote:
>> I then started a run at 96 clients which I accidentally killed shortly
>> before it was scheduled to finish, but the results are not much
>> different; there is no hint of the runaway CLogControlLock contention
>> that Dilip sees on power2.
>
On 10/20/2016 07:59 PM, Robert Haas wrote:
On Thu, Oct 20, 2016 at 11:45 AM, Robert Haas wrote:
On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote:
On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote:
>>
...
So here's my theory. The whole reason why Tomas is having difficulty
seeing any
On Thu, Oct 20, 2016 at 11:45 AM, Robert Haas wrote:
> On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote:
>> On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote:
>>> I agree with these conclusions. I had a chance to talk with Andres
>>> this morning at Postgres Vision and based on that convers
On Thu, Oct 20, 2016 at 3:36 AM, Dilip Kumar wrote:
> On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote:
>> I agree with these conclusions. I had a chance to talk with Andres
>> this morning at Postgres Vision and based on that conversation I'd
>> like to suggest a couple of additional tests:
On 10/20/2016 09:36 AM, Dilip Kumar wrote:
On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote:
I agree with these conclusions. I had a chance to talk with Andres
this morning at Postgres Vision and based on that conversation I'd
like to suggest a couple of additional tests:
1. Repeat this te
On Thu, Oct 13, 2016 at 12:25 AM, Robert Haas wrote:
> I agree with these conclusions. I had a chance to talk with Andres
> this morning at Postgres Vision and based on that conversation I'd
> like to suggest a couple of additional tests:
>
> 1. Repeat this test on x86. In particular, I think yo
On Thu, Oct 13, 2016 at 7:53 AM, Tomas Vondra
wrote:
> On 10/12/2016 08:55 PM, Robert Haas wrote:
>> On Wed, Oct 12, 2016 at 3:21 AM, Dilip Kumar wrote:
>>> I think at higher client count from client count 96 onwards contention
>>> on CLogControlLock is clearly visible and which is completely sol
On 10/12/2016 08:55 PM, Robert Haas wrote:
> On Wed, Oct 12, 2016 at 3:21 AM, Dilip Kumar wrote:
>> I think at higher client count from client count 96 onwards contention
>> on CLogControlLock is clearly visible and which is completely solved
>> with group lock patch.
>>
>> And at lower client cou
On Wed, Oct 12, 2016 at 3:21 AM, Dilip Kumar wrote:
> I think at higher client count from client count 96 onwards contention
> on CLogControlLock is clearly visible and which is completely solved
> with group lock patch.
>
> And at lower client count 32,64 contention on CLogControlLock is not
> s
On Mon, Oct 10, 2016 at 2:17 AM, Tomas Vondra
wrote:
> after testing each combination (every ~9 hours). Inspired by Robert's wait
> event post a few days ago, I've added wait event sampling so that we can
> perform similar analysis. (Neat idea!)
I have done wait event test on for head vs group lo
On 10/08/2016 07:47 AM, Amit Kapila wrote:
On Fri, Oct 7, 2016 at 3:02 PM, Tomas Vondra
wrote:
>
> ...
>
In total, I plan to test combinations of:
(a) Dilip's workload and pgbench (regular and -N)
(b) logged and unlogged tables
(c) scale 300 and scale 3000 (both fits into RAM)
(d) sync_commit
On Fri, Oct 7, 2016 at 3:02 PM, Tomas Vondra
wrote:
>
> I got access to a large machine with 72/144 cores (thanks to Oleg and
> Alexander from Postgres Professional), and I'm running the tests on that
> machine too.
>
> Results from Dilip's workload (with scale 300, unlogged tables) look like
> th
On 10/05/2016 10:03 AM, Amit Kapila wrote:
On Wed, Oct 5, 2016 at 12:05 PM, Tomas Vondra
wrote:
Hi,
After collecting a lot more results from multiple kernel versions, I can
confirm that I see a significant improvement with 128 and 192 clients,
roughly by 30%:
64
On Wed, Oct 5, 2016 at 12:05 PM, Tomas Vondra
wrote:
> Hi,
>
> After collecting a lot more results from multiple kernel versions, I can
> confirm that I see a significant improvement with 128 and 192 clients,
> roughly by 30%:
>
>64128192
> -
Hi,
After collecting a lot more results from multiple kernel versions, I can
confirm that I see a significant improvement with 128 and 192 clients,
roughly by 30%:
64128192
master
On Thu, Sep 29, 2016 at 8:05 PM, Robert Haas wrote:
> OK, another theory: Dilip is, I believe, reinitializing for each run,
> and you are not.
Yes, I am reinitializing for each run.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgs
On Thu, Sep 29, 2016 at 10:14 AM, Tomas Vondra
wrote:
>> It's not impossible that the longer runs could matter - performance
>> isn't necessarily stable across time during a pgbench test, and the
>> longer the run the more CLOG pages it will fill.
>
> Sure, but I'm not doing just a single pgbench
On 09/29/2016 03:47 PM, Robert Haas wrote:
On Wed, Sep 28, 2016 at 9:10 PM, Tomas Vondra
wrote:
I feel like we must be missing something here. If Dilip is seeing
huge speedups and you're seeing nothing, something is different, and
we don't know what it is. Even if the test case is artificial,
On Wed, Sep 28, 2016 at 9:10 PM, Tomas Vondra
wrote:
>> I feel like we must be missing something here. If Dilip is seeing
>> huge speedups and you're seeing nothing, something is different, and
>> we don't know what it is. Even if the test case is artificial, it
>> ought to be the same when one
On Thu, Sep 29, 2016 at 12:56 PM, Dilip Kumar wrote:
> On Thu, Sep 29, 2016 at 6:40 AM, Tomas Vondra
> wrote:
>> Yes, definitely - we're missing something important, I think. One difference
>> is that Dilip is using longer runs, but I don't think that's a problem (as I
>> demonstrated how stable
On Thu, Sep 29, 2016 at 6:40 AM, Tomas Vondra
wrote:
> Yes, definitely - we're missing something important, I think. One difference
> is that Dilip is using longer runs, but I don't think that's a problem (as I
> demonstrated how stable the results are).
>
> I wonder what CPU model is Dilip using
On 09/29/2016 01:59 AM, Robert Haas wrote:
On Wed, Sep 28, 2016 at 6:45 PM, Tomas Vondra
wrote:
So, is 300 too little? I don't think so, because Dilip saw some benefit from
that. Or what scale factor do we think is needed to reproduce the benefit?
My machine has 256GB of ram, so I can easily go
On Wed, Sep 28, 2016 at 6:45 PM, Tomas Vondra
wrote:
> So, is 300 too little? I don't think so, because Dilip saw some benefit from
> that. Or what scale factor do we think is needed to reproduce the benefit?
> My machine has 256GB of ram, so I can easily go up to 15000 and still keep
> everything
On 09/28/2016 05:39 PM, Robert Haas wrote:
On Tue, Sep 27, 2016 at 5:15 PM, Tomas Vondra
wrote:
So, I got the results from 3.10.101 (only the pgbench data), and it looks
like this:
3.10.101 1 8 16 32 64128192
-
On Tue, Sep 27, 2016 at 5:15 PM, Tomas Vondra
wrote:
> So, I got the results from 3.10.101 (only the pgbench data), and it looks
> like this:
>
> 3.10.101 1 8 16 32 64128192
>
> granular-l
On 09/26/2016 08:48 PM, Tomas Vondra wrote:
On 09/26/2016 07:16 PM, Tomas Vondra wrote:
The averages (over the 10 runs, 5 minute each) look like this:
3.2.80 1 8 16 32 64128192
granu
On Wed, Sep 21, 2016 at 8:47 AM, Dilip Kumar wrote:
> Summary:
> --
> At 32 clients no gain, I think at this workload Clog Lock is not a problem.
> At 64 Clients we can see ~10% gain with simple update and ~5% with TPCB.
> At 128 Clients we can see > 50% gain.
>
> Currently I have test
On 09/26/2016 07:16 PM, Tomas Vondra wrote:
The averages (over the 10 runs, 5 minute each) look like this:
3.2.80 1 8 16 32 64128192
granular-locking1567 12146 26341 44188 432
On Fri, Sep 23, 2016 at 9:20 AM, Amit Kapila wrote:
> On Fri, Sep 23, 2016 at 6:50 AM, Robert Haas wrote:
>> On Thu, Sep 22, 2016 at 7:44 PM, Tomas Vondra
>> wrote:
>>> I don't dare to suggest rejecting the patch, but I don't see how we could
>>> commit any of the patches at this point. So perha
On 09/24/2016 06:06 AM, Amit Kapila wrote:
On Fri, Sep 23, 2016 at 8:22 PM, Tomas Vondra
wrote:
...
>>
So I'm using 16GB shared buffers (so with scale 300 everything fits into
shared buffers), min_wal_size=16GB, max_wal_size=128GB, checkpoint timeout
1h etc. So no, there are no checkpoints du
On Fri, Sep 23, 2016 at 8:22 PM, Tomas Vondra
wrote:
> On 09/23/2016 03:07 PM, Amit Kapila wrote:
>>
>> On Fri, Sep 23, 2016 at 6:16 PM, Tomas Vondra
>> wrote:
>>>
>>> On 09/23/2016 01:44 AM, Tomas Vondra wrote:
...
The 4.5 kernel clearly changed the results significantly:
>>>
On 09/23/2016 02:59 PM, Pavan Deolasee wrote:
On Fri, Sep 23, 2016 at 6:05 PM, Tomas Vondra
mailto:tomas.von...@2ndquadrant.com>> wrote:
On 09/23/2016 05:10 AM, Amit Kapila wrote:
On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra
mailto:tomas.von...@2ndquadrant.com>> wrote:
On 09/23/2016 03:07 PM, Amit Kapila wrote:
On Fri, Sep 23, 2016 at 6:16 PM, Tomas Vondra
wrote:
On 09/23/2016 01:44 AM, Tomas Vondra wrote:
...
The 4.5 kernel clearly changed the results significantly:
...
(c) Although it's not visible in the results, 4.5.5 almost perfectly
eliminated th
On Fri, Sep 23, 2016 at 6:50 AM, Robert Haas wrote:
> On Thu, Sep 22, 2016 at 7:44 PM, Tomas Vondra
> wrote:
>> I don't dare to suggest rejecting the patch, but I don't see how we could
>> commit any of the patches at this point. So perhaps "returned with feedback"
>> and resubmitting in the next
On Fri, Sep 23, 2016 at 6:29 PM, Pavan Deolasee
wrote:
> On Fri, Sep 23, 2016 at 6:05 PM, Tomas Vondra
> wrote:
>>
>> On 09/23/2016 05:10 AM, Amit Kapila wrote:
>>>
>>> On Fri, Sep 23, 2016 at 5:14 AM, Tomas Vondra
>>> wrote:
On 09/21/2016 08:04 AM, Amit Kapila wrote:
>
>
1 - 100 of 245 matches
Mail list logo
