Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, May 07, 2016 at 08:25:01AM -0700, Paul E. McKenney wrote: > On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: > > On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney > > wrote: [ . . . ] > > Thought i would update a few runs with the linux-4.6-rc

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, May 07, 2016 at 08:25:01AM -0700, Paul E. McKenney wrote: > On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: > > On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney > > wrote: [ . . . ] > > Thought i would update a few runs with the linux-4.6-rc kernels. > > > > I have attached

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, May 8, 2016 at 1:25 AM, Paul E. McKenney wrote: > On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: >> On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney >> wrote: >> > On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E.

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, May 8, 2016 at 1:25 AM, Paul E. McKenney wrote: > On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: >> On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney >> wrote: >> > On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E. McKenney wrote: >> >> On Wed, Mar 30, 2016 at 07:55:47AM

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: > On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney > wrote: > > On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E. McKenney wrote: > >> On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, May 06, 2016 at 04:25:16PM +1000, Ross Green wrote: > On Sun, Apr 3, 2016 at 6:18 PM, Paul E. McKenney > wrote: > > On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E. McKenney wrote: > >> On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > >> > On Tue, Mar 29, 2016 at

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E. McKenney wrote: > On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > > On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Thu, Mar 31, 2016 at 08:42:55AM -0700, Paul E. McKenney wrote: > On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > > On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 30, 2016 at 07:55:47AM -0700, Paul E. McKenney wrote: > On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 29, 2016 at 06:49:08AM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > > > On Mon, Mar

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > > > On Sun, Mar 27,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 05:28:14PM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > > > On Sun, Mar 27,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > [ . . . ] > > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 05:25:18PM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > [ . . . ] > > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: [ . . . ] > > > OK, so I should instrument migration_call() if I get the repro rate up? > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 06:08:41AM -0700, Paul E. McKenney wrote: > On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: [ . . . ] > > > OK, so I should instrument migration_call() if I get the repro rate up? > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 04:12:19PM +, Mathieu Desnoyers wrote: > - On Mar 28, 2016, at 11:56 AM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: > >> - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney > >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 04:12:19PM +, Mathieu Desnoyers wrote: > - On Mar 28, 2016, at 11:56 AM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: > >> - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney > >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 11:56 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: >> - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney >> paul...@linux.vnet.ibm.com >> wrote: >> >> > On Mon, Mar 28, 2016 at 08:28:51AM

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 11:56 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: >> - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney >> paul...@linux.vnet.ibm.com >> wrote: >> >> > On Mon, Mar 28, 2016 at 08:28:51AM

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: > - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: > >> On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 03:07:36PM +, Mathieu Desnoyers wrote: > - On Mar 28, 2016, at 9:29 AM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: > >> On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 9:29 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: >> On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: >> >> > > Does that system have MONITOR/MWAIT errata? >> > >> > On the

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 9:29 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: >> On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: >> >> > > Does that system have MONITOR/MWAIT errata? >> > >> > On the

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 2:13 AM, Peter Zijlstra pet...@infradead.org wrote: > On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > >> >> But, you need hotplug for this to happen, right? >> > >> > My understanding is that this seems to be detection of failures to be >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 28, 2016, at 2:13 AM, Peter Zijlstra pet...@infradead.org wrote: > On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > >> >> But, you need hotplug for this to happen, right? >> > >> > My understanding is that this seems to be detection of failures to be >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:13:51AM +0200, Peter Zijlstra wrote: > On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > > > >> But, you need hotplug for this to happen, right? > > > > > > My understanding is that this seems to be detection of failures to be > > > awakened for a

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:13:51AM +0200, Peter Zijlstra wrote: > On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > > > >> But, you need hotplug for this to happen, right? > > > > > > My understanding is that this seems to be detection of failures to be > > > awakened for a

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: > > > > Does that system have MONITOR/MWAIT errata? > > > > On the off-chance that this question was also directed at me, > > Hehe, it wasn't, however, since we're

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:28:51AM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: > > > > Does that system have MONITOR/MWAIT errata? > > > > On the off-chance that this question was also directed at me, > > Hehe, it wasn't, however, since we're

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > > > But, you need hotplug for this to happen, right? > > > > I do, but Ross Green is seeing something that looks similar, and without > > CPU hotplug. > > Yes,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 08:25:47AM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > > > But, you need hotplug for this to happen, right? > > > > I do, but Ross Green is seeing something that looks similar, and without > > CPU hotplug. > > Yes,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: > > Does that system have MONITOR/MWAIT errata? > > On the off-chance that this question was also directed at me, Hehe, it wasn't, however, since we're here.. > here is > what I am running on. I am running in a qemu/KVM virtual

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 02:09:14PM -0700, Paul E. McKenney wrote: > > Does that system have MONITOR/MWAIT errata? > > On the off-chance that this question was also directed at me, Hehe, it wasn't, however, since we're here.. > here is > what I am running on. I am running in a qemu/KVM virtual

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > But, you need hotplug for this to happen, right? > > I do, but Ross Green is seeing something that looks similar, and without > CPU hotplug. Yes, but that's two differences so far, you need hotplug and he's on ARM (which

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 02:06:41PM -0700, Paul E. McKenney wrote: > > But, you need hotplug for this to happen, right? > > I do, but Ross Green is seeing something that looks similar, and without > CPU hotplug. Yes, but that's two differences so far, you need hotplug and he's on ARM (which

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > >> But, you need hotplug for this to happen, right? > > > > My understanding is that this seems to be detection of failures to be > > awakened for a long time on idle CPUs. It therefore seems to be more > > idle-related than

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 28, 2016 at 02:23:45AM +, Mathieu Desnoyers wrote: > >> But, you need hotplug for this to happen, right? > > > > My understanding is that this seems to be detection of failures to be > > awakened for a long time on idle CPUs. It therefore seems to be more > > idle-related than

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 27, 2016, at 9:44 PM, Mathieu Desnoyers mathieu.desnoy...@efficios.com wrote: > - On Mar 27, 2016, at 4:45 PM, Peter Zijlstra pet...@infradead.org wrote: > >> On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: >>> Oh, and the patch I am running with is below. I

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 27, 2016, at 9:44 PM, Mathieu Desnoyers mathieu.desnoy...@efficios.com wrote: > - On Mar 27, 2016, at 4:45 PM, Peter Zijlstra pet...@infradead.org wrote: > >> On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: >>> Oh, and the patch I am running with is below. I

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 27, 2016, at 4:45 PM, Peter Zijlstra pet...@infradead.org wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: >> Oh, and the patch I am running with is below. I am running x86, and so >> some other architectures would of course need the corresponding patch >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 27, 2016, at 4:45 PM, Peter Zijlstra pet...@infradead.org wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: >> Oh, and the patch I am running with is below. I am running x86, and so >> some other architectures would of course need the corresponding patch >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:54:39PM +0200, Peter Zijlstra wrote: > On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > > > > We're seeing a similar stall (~60 seconds) on an x86 development > > > > system here. Any luck tracking down the cause of this? If not, any > > > > suggestions for

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:54:39PM +0200, Peter Zijlstra wrote: > On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > > > > We're seeing a similar stall (~60 seconds) on an x86 development > > > > system here. Any luck tracking down the cause of this? If not, any > > > > suggestions for

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:53:18PM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > > Oh, and the patch I am running with is below. I am running x86, and so > > some other architectures would of course need the corresponding patch > > on that

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:53:18PM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > > Oh, and the patch I am running with is below. I am running x86, and so > > some other architectures would of course need the corresponding patch > > on that

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:45:59PM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > > Oh, and the patch I am running with is below. I am running x86, and so > > some other architectures would of course need the corresponding patch > > on that

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 10:45:59PM +0200, Peter Zijlstra wrote: > On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > > Oh, and the patch I am running with is below. I am running x86, and so > > some other architectures would of course need the corresponding patch > > on that

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > > > We're seeing a similar stall (~60 seconds) on an x86 development > > > system here. Any luck tracking down the cause of this? If not, any > > > suggestions for traces that might be helpful? > +Reinette, she has the system that can

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > > > We're seeing a similar stall (~60 seconds) on an x86 development > > > system here. Any luck tracking down the cause of this? If not, any > > > suggestions for traces that might be helpful? > +Reinette, she has the system that can

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > Oh, and the patch I am running with is below. I am running x86, and so > some other architectures would of course need the corresponding patch > on that architecture. > -#define TIF_POLLING_NRFLAG 21 /* idle is polling

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > Oh, and the patch I am running with is below. I am running x86, and so > some other architectures would of course need the corresponding patch > on that architecture. > -#define TIF_POLLING_NRFLAG 21 /* idle is polling

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > Oh, and the patch I am running with is below. I am running x86, and so > some other architectures would of course need the corresponding patch > on that architecture. > -#define TIF_POLLING_NRFLAG 21 /* idle is polling

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > Oh, and the patch I am running with is below. I am running x86, and so > some other architectures would of course need the corresponding patch > on that architecture. > -#define TIF_POLLING_NRFLAG 21 /* idle is polling

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > On Sun, Mar 27, 2016 at 01:48:55PM +, Mathieu Desnoyers wrote: > > - On Mar 26, 2016, at 9:34 PM, Paul E. McKenney > > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 08:40:18AM -0700, Paul E. McKenney wrote: > On Sun, Mar 27, 2016 at 01:48:55PM +, Mathieu Desnoyers wrote: > > - On Mar 26, 2016, at 9:34 PM, Paul E. McKenney > > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 01:48:55PM +, Mathieu Desnoyers wrote: > - On Mar 26, 2016, at 9:34 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: > >> - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney > >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sun, Mar 27, 2016 at 01:48:55PM +, Mathieu Desnoyers wrote: > - On Mar 26, 2016, at 9:34 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: > >> - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney > >>

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 26, 2016, at 9:34 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: >> - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney >> paul...@linux.vnet.ibm.com >> wrote: >> >> > On Sat, Mar 26, 2016 at 08:28:16AM

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 26, 2016, at 9:34 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: >> - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney >> paul...@linux.vnet.ibm.com >> wrote: >> >> > On Sat, Mar 26, 2016 at 08:28:16AM

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: > - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: > >> On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 10:22:57PM +, Mathieu Desnoyers wrote: > - On Mar 26, 2016, at 2:49 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: > >> On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 26, 2016, at 2:49 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: >> On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: >> > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 26, 2016, at 2:49 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: >> On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: >> > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: > On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: > > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney > > paul...@linux.vnet.ibm.com wrote: > > > > > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 08:28:16AM -0700, Paul E. McKenney wrote: > On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: > > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney > > paul...@linux.vnet.ibm.com wrote: > > > > > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: > >> Hi Paul, > >> > >> On 2016-03-23, Paul E. McKenney wrote: >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Sat, Mar 26, 2016 at 12:29:31PM +, Mathieu Desnoyers wrote: > - On Mar 25, 2016, at 5:46 PM, Paul E. McKenney > paul...@linux.vnet.ibm.com wrote: > > > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: > >> Hi Paul, > >> > >> On 2016-03-23, Paul E. McKenney wrote: >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 25, 2016, at 5:46 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: >> Hi Paul, >> >> On 2016-03-23, Paul E. McKenney wrote: >> > Please boot with the following parameters: >> > >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

- On Mar 25, 2016, at 5:46 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: > On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: >> Hi Paul, >> >> On 2016-03-23, Paul E. McKenney wrote: >> > Please boot with the following parameters: >> > >> >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-23, Paul E. McKenney wrote: > > Please boot with the following parameters: > > > > rcu_tree.rcu_kick_kthreads ftrace > > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > With

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 25, 2016 at 09:24:14PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-23, Paul E. McKenney wrote: > > Please boot with the following parameters: > > > > rcu_tree.rcu_kick_kthreads ftrace > > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > With

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-23, Paul E. McKenney wrote: > Please boot with the following parameters: > > rcu_tree.rcu_kick_kthreads ftrace > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi With these parameters I expected more details to show up in the kernel logs but cannot

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-23, Paul E. McKenney wrote: > Please boot with the following parameters: > > rcu_tree.rcu_kick_kthreads ftrace > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi With these parameters I expected more details to show up in the kernel logs but cannot

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 23, 2016 at 06:25:50PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-23, Paul E. McKenney wrote: > > Please boot with the following parameters: > > > > rcu_tree.rcu_kick_kthreads ftrace > > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > > > Or

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 23, 2016 at 06:25:50PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-23, Paul E. McKenney wrote: > > Please boot with the following parameters: > > > > rcu_tree.rcu_kick_kthreads ftrace > > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > > > Or

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-23, Paul E. McKenney wrote: > Please boot with the following parameters: > > rcu_tree.rcu_kick_kthreads ftrace > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > Or was this run with tracing? If so, less than three hours isn't too bad. This was

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-23, Paul E. McKenney wrote: > Please boot with the following parameters: > > rcu_tree.rcu_kick_kthreads ftrace > trace_event=sched_waking,sched_wakeup,sched_wake_idle_without_ipi > > Or was this run with tracing? If so, less than three hours isn't too bad. This was

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 23, 2016 at 05:15:11PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-22, Paul E. McKenney wrote: > > On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: > >> On 2016-03-22, Paul E. McKenney wrote: > >>> You set CONFIG_RCU_CPU_STALL_TIMEOUT=60, which matches

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Wed, Mar 23, 2016 at 05:15:11PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-22, Paul E. McKenney wrote: > > On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: > >> On 2016-03-22, Paul E. McKenney wrote: > >>> You set CONFIG_RCU_CPU_STALL_TIMEOUT=60, which matches

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-22, Paul E. McKenney wrote: > On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: >> On 2016-03-22, Paul E. McKenney wrote: >>> You set CONFIG_RCU_CPU_STALL_TIMEOUT=60, which matches the 60004 >>> jiffies above. Is that value due to a distro setting or

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-22, Paul E. McKenney wrote: > On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: >> On 2016-03-22, Paul E. McKenney wrote: >>> You set CONFIG_RCU_CPU_STALL_TIMEOUT=60, which matches the 60004 >>> jiffies above. Is that value due to a distro setting or

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-22, Paul E. McKenney wrote: > > On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: > >> On 2016-03-21, Paul E. McKenney wrote: > >>> On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 22, 2016 at 09:04:47PM +, Chatre, Reinette wrote: > Hi Paul, > > On 2016-03-22, Paul E. McKenney wrote: > > On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: > >> On 2016-03-21, Paul E. McKenney wrote: > >>> On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-22, Paul E. McKenney wrote: > On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: >> On 2016-03-21, Paul E. McKenney wrote: >>> On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: On Fri, 18 Mar 2016 16:56:41 -0700 "Paul E. McKenney"

RE: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

Hi Paul, On 2016-03-22, Paul E. McKenney wrote: > On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: >> On 2016-03-21, Paul E. McKenney wrote: >>> On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: On Fri, 18 Mar 2016 16:56:41 -0700 "Paul E. McKenney" wrote:

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: > Hi Paul, Hello, Reinette! > On 2016-03-21, Paul E. McKenney wrote: > > On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > >> On Fri, 18 Mar 2016 16:56:41 -0700 > >> "Paul E. McKenney"

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Tue, Mar 22, 2016 at 04:35:32PM +, Chatre, Reinette wrote: > Hi Paul, Hello, Reinette! > On 2016-03-21, Paul E. McKenney wrote: > > On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > >> On Fri, 18 Mar 2016 16:56:41 -0700 > >> "Paul E. McKenney" wrote: > >>> On Fri, Mar 18, 2016

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > On Fri, 18 Mar 2016 16:56:41 -0700 > "Paul E. McKenney" wrote: > > On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > > > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: [ . .

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Mon, Mar 21, 2016 at 09:22:30AM -0700, Jacob Pan wrote: > On Fri, 18 Mar 2016 16:56:41 -0700 > "Paul E. McKenney" wrote: > > On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > > > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: [ . . . ] > > > We're seeing a

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, 18 Mar 2016 16:56:41 -0700 "Paul E. McKenney" wrote: > On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > > > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, 18 Mar 2016 16:56:41 -0700 "Paul E. McKenney" wrote: > On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > > > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > > > On Wed, Feb 24, 2016 at

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > On Wed, Feb 24, 2016 at 8:28 AM, Ross Green wrote: > > > On Wed, Feb 24, 2016 at 7:55 AM, Paul E. McKenney > > >

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > On Wed, Feb 24, 2016 at 8:28 AM, Ross Green wrote: > > > On Wed, Feb 24, 2016 at 7:55 AM, Paul E. McKenney > > > wrote: > > [ . . . ] > > > >> Still working on

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > > On Wed, Feb 24, 2016 at 8:28 AM, Ross Green wrote: > > > > On Wed, Feb 24,

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 18, 2016 at 02:00:11PM -0700, Josh Triplett wrote: > On Thu, Feb 25, 2016 at 04:56:38PM -0800, Paul E. McKenney wrote: > > On Thu, Feb 25, 2016 at 04:13:11PM +1100, Ross Green wrote: > > > On Wed, Feb 24, 2016 at 8:28 AM, Ross Green wrote: > > > > On Wed, Feb 24, 2016 at 7:55 AM, Paul

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 04, 2016 at 04:30:12PM +1100, Ross Green wrote: > On Fri, Feb 26, 2016 at 12:35 PM, Paul E. McKenney > wrote: [ . . . ] > >> OK, so what wakeup path omits the sched_wakeup event? > >> > >> The sched_waking event looks to occur once in try_to_wake_up()

Re: rcu_preempt self-detected stall on CPU from 4.5-rc3, since 3.17

On Fri, Mar 04, 2016 at 04:30:12PM +1100, Ross Green wrote: > On Fri, Feb 26, 2016 at 12:35 PM, Paul E. McKenney > wrote: [ . . . ] > >> OK, so what wakeup path omits the sched_wakeup event? > >> > >> The sched_waking event looks to occur once in try_to_wake_up() and > >> once in

  1   2   >