Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-22 Thread Heikki Linnakangas
On 21.11.2013 21:37, Merlin Moncure wrote: On Thu, Nov 21, 2013 at 10:37 AM, Heikki Linnakangas In my patch, I put the barrier inside the if (!LocalRecoveryInProgress) block. That codepath can only execute once in a backend, so performance is not an issue there. Does that look sane to you? oh

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Merlin Moncure
On Thu, Nov 21, 2013 at 10:37 AM, Heikki Linnakangas wrote: > On 21.11.2013 17:08, Merlin Moncure wrote: >> >> On Thu, Nov 21, 2013 at 9:02 AM, Andres Freund >> wrote: >>> >>> On 2013-11-21 16:25:02 +0200, Heikki Linnakangas wrote: Hmm. All callers of RecoveryInProgress() must be prepar

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Heikki Linnakangas
On 21.11.2013 17:08, Merlin Moncure wrote: On Thu, Nov 21, 2013 at 9:02 AM, Andres Freund wrote: On 2013-11-21 16:25:02 +0200, Heikki Linnakangas wrote: Hmm. All callers of RecoveryInProgress() must be prepared to handle the case that RecoveryInProgress() returns true, but the system is no lon

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Merlin Moncure
On Thu, Nov 21, 2013 at 9:02 AM, Andres Freund wrote: > On 2013-11-21 16:25:02 +0200, Heikki Linnakangas wrote: >> Hmm. All callers of RecoveryInProgress() must be prepared to handle the case >> that RecoveryInProgress() returns true, but the system is no longer in >> recovery. No matter what lock

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Andres Freund
On 2013-11-21 16:25:02 +0200, Heikki Linnakangas wrote: > Hmm. All callers of RecoveryInProgress() must be prepared to handle the case > that RecoveryInProgress() returns true, but the system is no longer in > recovery. No matter what locking we do in RecoveryInProgress(), the startup > process mig

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Merlin Moncure
On Thu, Nov 21, 2013 at 9:09 AM, Andres Freund wrote: > On 2013-11-21 09:08:05 -0600, Merlin Moncure wrote: >> This code is in a very hot code path. Are we *sure* that the read >> barrier is fast enough that we don't want to provide an alternate >> function that only returns the local flag? I do

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Andres Freund
On 2013-11-21 09:08:05 -0600, Merlin Moncure wrote: > This code is in a very hot code path. Are we *sure* that the read > barrier is fast enough that we don't want to provide an alternate > function that only returns the local flag? I don't know enough about > them to say either way. A read barr

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-11-21 Thread Heikki Linnakangas
On 03.10.2013 00:14, Merlin Moncure wrote: On Wed, Oct 2, 2013 at 3:43 PM, Ants Aasma wrote: On Wed, Oct 2, 2013 at 10:37 PM, Merlin Moncure wrote: On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma wrote: I haven't reviewed the code in as much detail to say if there is an actual race here, I tend

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Merlin Moncure
On Wed, Oct 2, 2013 at 3:43 PM, Ants Aasma wrote: > On Wed, Oct 2, 2013 at 10:37 PM, Merlin Moncure wrote: >> On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma wrote: >>> I haven't reviewed the code in as much detail to say if there is an >>> actual race here, I tend to think there's probably not, but

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Ants Aasma
On Wed, Oct 2, 2013 at 10:37 PM, Merlin Moncure wrote: > On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma wrote: >> I haven't reviewed the code in as much detail to say if there is an >> actual race here, I tend to think there's probably not, but the >> specific pattern that I had in mind is that with

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Merlin Moncure
On Wed, Oct 2, 2013 at 9:45 AM, Ants Aasma wrote: > I haven't reviewed the code in as much detail to say if there is an > actual race here, I tend to think there's probably not, but the > specific pattern that I had in mind is that with the following actual > code: hm. I think there *is* a race.

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Ants Aasma
On Wed, Oct 2, 2013 at 4:39 PM, Merlin Moncure wrote: > On Mon, Sep 30, 2013 at 7:51 PM, Ants Aasma wrote: >> So we need a read barrier somewhere *after* reading the flag in >> RecoveryInProgress() and reading the shared memory structures, and in >> theory a full barrier if we are going to be wri

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Andres Freund
On 2013-10-02 08:39:59 -0500, Merlin Moncure wrote: > wow -- thanks for your review and provided detail. Considering there > are no examples of barrier instructions to date, I think some of your > commentary should be included in the in-source documentation. I think most of it is in README.barrie

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-02 Thread Merlin Moncure
On Mon, Sep 30, 2013 at 8:15 AM, Peter Eisentraut wrote: > On 9/27/13 3:00 PM, Merlin Moncure wrote: >> Attached is simplified patch that replaces the spinlock with a read >> barrier based on a suggestion made by Andres offlist. > > This patch doesn't apply. works for me: merlin@mmoncure-ubuntu:

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-01 Thread Ants Aasma
On Tue, Oct 1, 2013 at 2:08 PM, Andres Freund wrote: >> As for the specific patch being discussed here. The read barrier is in >> the wrong place and with the wrong comment, and the write side is >> assuming that SpinLockAcquire() is a write barrier, which it isn't >> documented to do at the momen

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-01 Thread Andres Freund
On 2013-10-01 14:31:11 +0300, Ants Aasma wrote: > >> The correct way to think of this is > >> that StartupXLOG() does a bunch of state modifications and then > >> advertises the fact that it's done by setting > >> xlogctl->SharedRecoveryInProgress = false; The state modifications > >> should better

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-10-01 Thread Andres Freund
On 2013-10-01 03:51:50 +0300, Ants Aasma wrote: > On Sat, Sep 28, 2013 at 12:53 AM, Andres Freund > wrote: > > What confuses me is that pg_read_barrier() is just a compiler barrier on > > x86[-64] in barrier.h. According to my knowledge it needs to be an > > lfence or the full barrier? > > The li

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-30 Thread Ants Aasma
On Sat, Sep 28, 2013 at 12:53 AM, Andres Freund wrote: > What confuses me is that pg_read_barrier() is just a compiler barrier on > x86[-64] in barrier.h. According to my knowledge it needs to be an > lfence or the full barrier? > The linked papers from Paul McKenney - which are a great read - see

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-30 Thread Peter Eisentraut
On 9/27/13 3:00 PM, Merlin Moncure wrote: > Attached is simplified patch that replaces the spinlock with a read > barrier based on a suggestion made by Andres offlist. This patch doesn't apply. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Andres Freund
Hi, What confuses me is that pg_read_barrier() is just a compiler barrier on x86[-64] in barrier.h. According to my knowledge it needs to be an lfence or the full barrier? The linked papers from Paul McKenney - which are a great read - seem to agree? On 2013-09-27 23:12:17 +0200, Andres Freund wr

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Andres Freund
On 2013-09-27 23:28:37 +0300, Heikki Linnakangas wrote: > On 27.09.2013 22:00, Merlin Moncure wrote: > >Attached is simplified patch that replaces the spinlock with a read > >barrier based on a suggestion made by Andres offlist. The approach > >has different performance characteristics -- a barrie

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Peter Geoghegan
On Fri, Sep 27, 2013 at 1:28 PM, Heikki Linnakangas wrote: > Yep. I only added the first caller of the barriers altogether in the > xlog-insertion scaling patch. Robert wrote the infrastructure in 9.3, but it > wasn't used until now, in 9.4. FWIW, it was actually during 9.2 development that Rober

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Heikki Linnakangas
On 27.09.2013 22:00, Merlin Moncure wrote: On Fri, Sep 27, 2013 at 7:56 AM, Merlin Moncure wrote: On Thu, Sep 26, 2013 at 10:14 PM, Merlin Moncure wrote: On Thu, Sep 26, 2013 at 6:08 PM, Andres Freund wrote: On 2013-08-27 12:17:55 -0500, Merlin Moncure wrote: On Tue, Aug 27, 2013 at 10:55

Re: [HACKERS] [PERFORM] Cpu usage 100% on slave. s_lock problem.

2013-09-27 Thread Merlin Moncure
On Fri, Sep 27, 2013 at 7:56 AM, Merlin Moncure wrote: > On Thu, Sep 26, 2013 at 10:14 PM, Merlin Moncure wrote: >> On Thu, Sep 26, 2013 at 6:08 PM, Andres Freund >> wrote: >>> On 2013-08-27 12:17:55 -0500, Merlin Moncure wrote: On Tue, Aug 27, 2013 at 10:55 AM, Andres Freund wrote: