Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
On Thu, Mar 28, 2019 at 06:28:26AM +0100, Greg Kroah-Hartman wrote: > On Thu, Mar 28, 2019 at 01:36:12AM +, Zhang, Jun wrote: > > Hello, Paul > > > > Yes, I only use original V3.18.136 to test. Because system run very slowly, > > I give up. > > > > Device: NUC (made in 2017) > > OS:ubuntu 16.04 > > Kernel: V3.18.136 (come from > > https://kernel.ubuntu.com/~kernel-ppa/mainline/v3.18.136/) > > System can boot, but run very slowly, and couldn't connect the network. > > This make me not to work with terminal, because system very slowly. > > So I don’t use V3.18.136+patch to test. > > Please do not worry about 3.18.y if it is any problem, I only keep it > alive for a few users and if they have not reported issues yet, it is > fine. Please do not do any extra work for this kernel tree at all, it > is just too old :) Very good. ;-) >From what I can see, if 3.18.y works for them, then 3.18.y+patch should also work for them. Thanx, Paul
Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
On Thu, Mar 28, 2019 at 01:36:12AM +, Zhang, Jun wrote: > Hello, Paul > > Yes, I only use original V3.18.136 to test. Because system run very slowly, I > give up. > > Device: NUC (made in 2017) > OS:ubuntu 16.04 > Kernel: V3.18.136 (come from > https://kernel.ubuntu.com/~kernel-ppa/mainline/v3.18.136/) > System can boot, but run very slowly, and couldn't connect the network. > This make me not to work with terminal, because system very slowly. > So I don’t use V3.18.136+patch to test. Please do not worry about 3.18.y if it is any problem, I only keep it alive for a few users and if they have not reported issues yet, it is fine. Please do not do any extra work for this kernel tree at all, it is just too old :) thanks, greg k-h
RE: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
Hello, Paul Yes, I only use original V3.18.136 to test. Because system run very slowly, I give up. Device: NUC (made in 2017) OS:ubuntu 16.04 Kernel: V3.18.136 (come from https://kernel.ubuntu.com/~kernel-ppa/mainline/v3.18.136/) System can boot, but run very slowly, and couldn't connect the network. This make me not to work with terminal, because system very slowly. So I don’t use V3.18.136+patch to test. -Original Message- From: Paul E. McKenney [mailto:paul...@linux.ibm.com] Sent: Wednesday, March 27, 2019 23:10 To: Zhang, Jun Cc: He, Bo ; Greg Kroah-Hartman ; linux-kernel@vger.kernel.org; sta...@vger.kernel.org; Xiao, Jin ; Bai, Jie A Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt Hello, Jun, Do you see the same hang without the patch? Thanx, Paul On Wed, Mar 27, 2019 at 01:50:57AM +, Zhang, Jun wrote: > Hello，Paul > > I use a new NUC (made in 2017) to test, meet the same hang. System run very > slowly. > > But I use my PC (made before 2015), test V3.18.136+ patch for 12 hours, still > well. > > Maybe V3.18.y don't support some new devices. > > > -Original Message- > From: Paul E. McKenney [mailto:paul...@linux.ibm.com] > Sent: Tuesday, March 26, 2019 23:56 > To: He, Bo > Cc: Greg Kroah-Hartman ; > linux-kernel@vger.kernel.org; sta...@vger.kernel.org; Zhang, Jun > ; Xiao, Jin ; Bai, Jie A > > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup > from softirq and interrupt > > On Tue, Mar 26, 2019 at 08:43:45AM +, He, Bo wrote: > > Hi, Paul: > > I have tried on my PC and not hit any hang issue with RCU torture test > > for one hour, the configurations are like: > > OS: ubuntu 16.04 > > kenrel: 3.18.136 + 3.18 rcu patch > > CPU: Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz > > Sounds good, please proceed! > > Thanx, Paul > > > -Original Message- > > From: Paul E. McKenney > > Sent: Tuesday, March 26, 2019 12:00 AM > > To: Greg Kroah-Hartman > > Cc: He, Bo ; linux-kernel@vger.kernel.org; > > sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin > > ; Bai, Jie A > > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup > > from softirq and interrupt > > > > On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > > > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > > > Hi, Greg: > > > > Can you hold on the 3.18-stable branch, it seems there are some > > > > issue, please see the comments from Paul: > > > > > > > > Comments from Paul: > > > > I subjected all of the others to light rcutorture testing, which > > > > they passed. This v3.18 patch hung, however. Trying it again > > > > with stock > > > > v3.18 got the same hang, so I believe we can exonerate the patch and > > > > give it a good firm "maybe" on 3.18. > > > > > > > > Worth paying special attention to further test results from 3.18.x, > > > > though! > > > > > > Ok, I've dropped this from the 3.18.y queue now, thanks. > > > > Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture > > testing, please let me know. > > > > Thanx, Paul > > >
Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
Hello, Jun, Do you see the same hang without the patch? Thanx, Paul On Wed, Mar 27, 2019 at 01:50:57AM +, Zhang, Jun wrote: > Hello，Paul > > I use a new NUC (made in 2017) to test, meet the same hang. System run very > slowly. > > But I use my PC (made before 2015), test V3.18.136+ patch for 12 hours, still > well. > > Maybe V3.18.y don't support some new devices. > > > -Original Message- > From: Paul E. McKenney [mailto:paul...@linux.ibm.com] > Sent: Tuesday, March 26, 2019 23:56 > To: He, Bo > Cc: Greg Kroah-Hartman ; > linux-kernel@vger.kernel.org; sta...@vger.kernel.org; Zhang, Jun > ; Xiao, Jin ; Bai, Jie A > > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from > softirq and interrupt > > On Tue, Mar 26, 2019 at 08:43:45AM +, He, Bo wrote: > > Hi, Paul: > > I have tried on my PC and not hit any hang issue with RCU torture test > > for one hour, the configurations are like: > > OS: ubuntu 16.04 > > kenrel: 3.18.136 + 3.18 rcu patch > > CPU: Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz > > Sounds good, please proceed! > > Thanx, Paul > > > -Original Message- > > From: Paul E. McKenney > > Sent: Tuesday, March 26, 2019 12:00 AM > > To: Greg Kroah-Hartman > > Cc: He, Bo ; linux-kernel@vger.kernel.org; > > sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin > > ; Bai, Jie A > > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup > > from softirq and interrupt > > > > On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > > > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > > > Hi, Greg: > > > > Can you hold on the 3.18-stable branch, it seems there are some > > > > issue, please see the comments from Paul: > > > > > > > > Comments from Paul: > > > > I subjected all of the others to light rcutorture testing, which > > > > they passed. This v3.18 patch hung, however. Trying it again > > > > with stock > > > > v3.18 got the same hang, so I believe we can exonerate the patch and > > > > give it a good firm "maybe" on 3.18. > > > > > > > > Worth paying special attention to further test results from 3.18.x, > > > > though! > > > > > > Ok, I've dropped this from the 3.18.y queue now, thanks. > > > > Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture > > testing, please let me know. > > > > Thanx, Paul > > >
RE: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
Hello，Paul I use a new NUC (made in 2017) to test, meet the same hang. System run very slowly. But I use my PC (made before 2015), test V3.18.136+ patch for 12 hours, still well. Maybe V3.18.y don't support some new devices. -Original Message- From: Paul E. McKenney [mailto:paul...@linux.ibm.com] Sent: Tuesday, March 26, 2019 23:56 To: He, Bo Cc: Greg Kroah-Hartman ; linux-kernel@vger.kernel.org; sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin ; Bai, Jie A Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt On Tue, Mar 26, 2019 at 08:43:45AM +, He, Bo wrote: > Hi, Paul: > I have tried on my PC and not hit any hang issue with RCU torture test > for one hour, the configurations are like: > OS: ubuntu 16.04 > kenrel: 3.18.136 + 3.18 rcu patch > CPU: Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz Sounds good, please proceed! Thanx, Paul > -Original Message- > From: Paul E. McKenney > Sent: Tuesday, March 26, 2019 12:00 AM > To: Greg Kroah-Hartman > Cc: He, Bo ; linux-kernel@vger.kernel.org; > sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin > ; Bai, Jie A > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup > from softirq and interrupt > > On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > > Hi, Greg: > > > Can you hold on the 3.18-stable branch, it seems there are some issue, > > > please see the comments from Paul: > > > > > > Comments from Paul: > > > I subjected all of the others to light rcutorture testing, which > > > they passed. This v3.18 patch hung, however. Trying it again > > > with stock > > > v3.18 got the same hang, so I believe we can exonerate the patch and give > > > it a good firm "maybe" on 3.18. > > > > > > Worth paying special attention to further test results from 3.18.x, > > > though! > > > > Ok, I've dropped this from the 3.18.y queue now, thanks. > > Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture > testing, please let me know. > > Thanx, Paul >
Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
On Tue, Mar 26, 2019 at 08:43:45AM +, He, Bo wrote: > Hi, Paul: > I have tried on my PC and not hit any hang issue with RCU torture test > for one hour, the configurations are like: > OS: ubuntu 16.04 > kenrel: 3.18.136 + 3.18 rcu patch > CPU: Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz Sounds good, please proceed! Thanx, Paul > -Original Message- > From: Paul E. McKenney > Sent: Tuesday, March 26, 2019 12:00 AM > To: Greg Kroah-Hartman > Cc: He, Bo ; linux-kernel@vger.kernel.org; > sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin > ; Bai, Jie A > Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from > softirq and interrupt > > On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > > Hi, Greg: > > > Can you hold on the 3.18-stable branch, it seems there are some issue, > > > please see the comments from Paul: > > > > > > Comments from Paul: > > > I subjected all of the others to light rcutorture testing, which > > > they passed. This v3.18 patch hung, however. Trying it again with > > > stock > > > v3.18 got the same hang, so I believe we can exonerate the patch and give > > > it a good firm "maybe" on 3.18. > > > > > > Worth paying special attention to further test results from 3.18.x, > > > though! > > > > Ok, I've dropped this from the 3.18.y queue now, thanks. > > Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture > testing, please let me know. > > Thanx, Paul >
RE: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
Hi, Paul: I have tried on my PC and not hit any hang issue with RCU torture test for one hour, the configurations are like: OS: ubuntu 16.04 kenrel: 3.18.136 + 3.18 rcu patch CPU: Intel(R) Xeon(R) CPU E3-1225 V2 @ 3.20GHz -Original Message- From: Paul E. McKenney Sent: Tuesday, March 26, 2019 12:00 AM To: Greg Kroah-Hartman Cc: He, Bo ; linux-kernel@vger.kernel.org; sta...@vger.kernel.org; Zhang, Jun ; Xiao, Jin ; Bai, Jie A Subject: Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > Hi, Greg: > > Can you hold on the 3.18-stable branch, it seems there are some issue, > > please see the comments from Paul: > > > > Comments from Paul: > > I subjected all of the others to light rcutorture testing, which > > they passed. This v3.18 patch hung, however. Trying it again with > > stock > > v3.18 got the same hang, so I believe we can exonerate the patch and give > > it a good firm "maybe" on 3.18. > > > > Worth paying special attention to further test results from 3.18.x, though! > > Ok, I've dropped this from the 3.18.y queue now, thanks. Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture testing, please let me know. Thanx, Paul
Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
On Sat, Mar 23, 2019 at 07:33:15AM +0100, Greg Kroah-Hartman wrote: > On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > > Hi, Greg: > > Can you hold on the 3.18-stable branch, it seems there are some issue, > > please see the comments from Paul: > > > > Comments from Paul: > > I subjected all of the others to light rcutorture testing, which they > > passed. This v3.18 patch hung, however. Trying it again with stock > > v3.18 got the same hang, so I believe we can exonerate the patch and give > > it a good firm "maybe" on 3.18. > > > > Worth paying special attention to further test results from 3.18.x, though! > > Ok, I've dropped this from the 3.18.y queue now, thanks. Bo, if you know of a "y" for 3.18.y that would likely pass rcutorture testing, please let me know. Thanx, Paul
Re: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
On Fri, Mar 22, 2019 at 04:00:17PM +, He, Bo wrote: > Hi, Greg: > Can you hold on the 3.18-stable branch, it seems there are some issue, > please see the comments from Paul: > > Comments from Paul: > I subjected all of the others to light rcutorture testing, which they passed. > This v3.18 patch hung, however. Trying it again with stock > v3.18 got the same hang, so I believe we can exonerate the patch and give it > a good firm "maybe" on 3.18. > > Worth paying special attention to further test results from 3.18.x, though! Ok, I've dropped this from the 3.18.y queue now, thanks. greg k-h
RE: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq and interrupt
Hi, Greg:
Can you hold on the 3.18-stable branch, it seems there are some issue,
please see the comments from Paul:
Comments from Paul:
I subjected all of the others to light rcutorture testing, which they passed.
This v3.18 patch hung, however. Trying it again with stock
v3.18 got the same hang, so I believe we can exonerate the patch and give it a
good firm "maybe" on 3.18.
Worth paying special attention to further test results from 3.18.x, though!
-Original Message-
From: Greg Kroah-Hartman
Sent: Friday, March 22, 2019 7:16 PM
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman ; sta...@vger.kernel.org;
He, Bo ; Zhang, Jun ; Paul E. McKenney
; Xiao, Jin ; Bai, Jie A
Subject: [PATCH 3.18 132/134] rcu: Do RCU GP kthread self-wakeup from softirq
and interrupt
3.18-stable review patch. If anyone has any objections, please let me know.
--
From: Zhang, Jun
commit 1d1f898df6586c5ea9aeaf349f13089c6fa37903 upstream.
The rcu_gp_kthread_wake() function is invoked when it might be necessary to
wake the RCU grace-period kthread. Because self-wakeups are normally a useless
waste of CPU cycles, if rcu_gp_kthread_wake() is invoked from this kthread, it
naturally refuses to do the wakeup.
Unfortunately, natural though it might be, this heuristic fails when
rcu_gp_kthread_wake() is invoked from an interrupt or softirq handler that
interrupted the grace-period kthread just after the final check of the
wait-event condition but just before the schedule() call. In this case, a
wakeup is required, even though the call to rcu_gp_kthread_wake() is within the
RCU grace-period kthread's context. Failing to provide this wakeup can result
in grace periods failing to start, which in turn results in out-of-memory
conditions.
This race window is quite narrow, but it actually did happen during real
testing. It would of course need to be fixed even if it was strictly
theoretical in nature.
This patch does not Cc stable because it does not apply cleanly to earlier
kernel versions.
Fixes: 48a7639ce80c ("rcu: Make callers awaken grace-period kthread")
Reported-by: "He, Bo"
Co-developed-by: "Zhang, Jun"
Co-developed-by: "He, Bo"
Co-developed-by: "xiao, jin"
Co-developed-by: Bai, Jie A
Signed-off: "Zhang, Jun"
Signed-off: "He, Bo"
Signed-off: "xiao, jin"
Signed-off: Bai, Jie A
Signed-off-by: "Zhang, Jun" [ paulmck: Switch from
!in_softirq() to "!in_interrupt() &&
!in_serving_softirq() to avoid redundant wakeups and to also handle the
interrupt-handler scenario as well as the softirq-handler scenario that
actually occurred in testing. ]
Signed-off-by: Paul E. McKenney
Link:
https://lkml.kernel.org/r/cd6925e8781efd4d8e11882d20fc406d52a11...@shsmsx104.ccr.corp.intel.com
Signed-off-by: Greg Kroah-Hartman
---
kernel/rcu/tree.c | 20 ++--
1 file changed, 14 insertions(+), 6 deletions(-)
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1399,15 +1399,23 @@ static int rcu_future_gp_cleanup(struct }
/*
- * Awaken the grace-period kthread for the specified flavor of RCU.
- * Don't do a self-awaken, and don't bother awakening when there is
- * nothing for the grace-period kthread to do (as in several CPUs
- * raced to awaken, and we lost), and finally don't try to awaken
- * a kthread that has not yet been created.
+ * Awaken the grace-period kthread. Don't do a self-awaken (unless in
+ * an interrupt or softirq handler), and don't bother awakening when
+ there
+ * is nothing for the grace-period kthread to do (as in several CPUs
+ raced
+ * to awaken, and we lost), and finally don't try to awaken a kthread
+ that
+ * has not yet been created. If all those checks are passed, track
+ some
+ * debug information and awaken.
+ *
+ * So why do the self-wakeup when in an interrupt or softirq handler
+ * in the grace-period kthread's context? Because the kthread might
+ have
+ * been interrupted just as it was going to sleep, and just after the
+ final
+ * pre-sleep check of the awaken condition. In this case, a wakeup
+ really
+ * is required, and is therefore supplied.
*/
static void rcu_gp_kthread_wake(struct rcu_state *rsp) {
- if (current == rsp->gp_kthread ||
+ if ((current == rsp->gp_kthread &&
+!in_interrupt() && !in_serving_softirq()) ||
!ACCESS_ONCE(rsp->gp_flags) ||
!rsp->gp_kthread)
return;
