Re: [PATCH 12/13] rcu/nocb: Prepare for finegrained deferred wakeup
On Tue, Mar 16, 2021 at 12:45:26PM +0100, Frederic Weisbecker wrote:
> On Mon, Mar 15, 2021 at 08:02:39PM -0700, Paul E. McKenney wrote:
> > On Tue, Feb 23, 2021 at 01:10:10AM +0100, Frederic Weisbecker wrote:
> > > Provide a way to tune the deferred wakeup level we want to perform from
> > > a safe wakeup point. Currently those sites are:
> > >
> > > * nocb_timer
> > > * user/idle/guest entry
> > > * CPU down
> > > * softirq/rcuc
> > >
> > > All of these sites perform the wake up for both RCU_NOCB_WAKE and
> > > RCU_NOCB_WAKE_FORCE.
> > >
> > > In order to merge nocb_timer and nocb_bypass_timer together, we plan to
> > > add a new RCU_NOCB_WAKE_BYPASS that really want to be deferred until
> > > a timer fires so that we don't wake up the NOCB-gp kthread too early.
> > >
> > > To prepare for that, integrate a wake up level/limit that a callsite is
> > > deemed to perform.
> >
> > This appears to need the following in order to build for non-NOCB
> > configurations. I will fold it in and am retesting.
> >
> > Thanx, Paul
> >
> >
> >
> > commit 55f59dd75a11455cf558fd387fbf9011017dcc8a
> > Author: Paul E. McKenney
> > Date: Mon Mar 15 20:00:34 2021 -0700
> >
> > squash! rcu/nocb: Prepare for fine-grained deferred wakeup
> >
> > [ paulmck: Fix non-NOCB rcu_nocb_need_deferred_wakeup() definition. ]
> > Signed-off-by: Paul E. McKenney
> >
> > diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> > index 0cc7f68..dfb048e 100644
> > --- a/kernel/rcu/tree_plugin.h
> > +++ b/kernel/rcu/tree_plugin.h
> > @@ -2926,7 +2926,7 @@ static void __init
> > rcu_boot_init_nocb_percpu_data(struct rcu_data *rdp)
> > {
> > }
> >
> > -static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
> > +static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp, int level)
> > {
> > return false;
> > }
>
> Oops thanks, I missed that.
And with this, light rcutorture testing passed. I hope to pound on it
harder later this week.
Thanx, Paul
Re: [PATCH 12/13] rcu/nocb: Prepare for finegrained deferred wakeup
On Mon, Mar 15, 2021 at 08:02:39PM -0700, Paul E. McKenney wrote:
> On Tue, Feb 23, 2021 at 01:10:10AM +0100, Frederic Weisbecker wrote:
> > Provide a way to tune the deferred wakeup level we want to perform from
> > a safe wakeup point. Currently those sites are:
> >
> > * nocb_timer
> > * user/idle/guest entry
> > * CPU down
> > * softirq/rcuc
> >
> > All of these sites perform the wake up for both RCU_NOCB_WAKE and
> > RCU_NOCB_WAKE_FORCE.
> >
> > In order to merge nocb_timer and nocb_bypass_timer together, we plan to
> > add a new RCU_NOCB_WAKE_BYPASS that really want to be deferred until
> > a timer fires so that we don't wake up the NOCB-gp kthread too early.
> >
> > To prepare for that, integrate a wake up level/limit that a callsite is
> > deemed to perform.
>
> This appears to need the following in order to build for non-NOCB
> configurations. I will fold it in and am retesting.
>
> Thanx, Paul
>
>
>
> commit 55f59dd75a11455cf558fd387fbf9011017dcc8a
> Author: Paul E. McKenney
> Date: Mon Mar 15 20:00:34 2021 -0700
>
> squash! rcu/nocb: Prepare for fine-grained deferred wakeup
>
> [ paulmck: Fix non-NOCB rcu_nocb_need_deferred_wakeup() definition. ]
> Signed-off-by: Paul E. McKenney
>
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
> index 0cc7f68..dfb048e 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -2926,7 +2926,7 @@ static void __init
> rcu_boot_init_nocb_percpu_data(struct rcu_data *rdp)
> {
> }
>
> -static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
> +static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp, int level)
> {
> return false;
> }
Oops thanks, I missed that.
Re: [PATCH 12/13] rcu/nocb: Prepare for finegrained deferred wakeup
On Tue, Feb 23, 2021 at 01:10:10AM +0100, Frederic Weisbecker wrote:
> Provide a way to tune the deferred wakeup level we want to perform from
> a safe wakeup point. Currently those sites are:
>
> * nocb_timer
> * user/idle/guest entry
> * CPU down
> * softirq/rcuc
>
> All of these sites perform the wake up for both RCU_NOCB_WAKE and
> RCU_NOCB_WAKE_FORCE.
>
> In order to merge nocb_timer and nocb_bypass_timer together, we plan to
> add a new RCU_NOCB_WAKE_BYPASS that really want to be deferred until
> a timer fires so that we don't wake up the NOCB-gp kthread too early.
>
> To prepare for that, integrate a wake up level/limit that a callsite is
> deemed to perform.
This appears to need the following in order to build for non-NOCB
configurations. I will fold it in and am retesting.
Thanx, Paul
commit 55f59dd75a11455cf558fd387fbf9011017dcc8a
Author: Paul E. McKenney
Date: Mon Mar 15 20:00:34 2021 -0700
squash! rcu/nocb: Prepare for fine-grained deferred wakeup
[ paulmck: Fix non-NOCB rcu_nocb_need_deferred_wakeup() definition. ]
Signed-off-by: Paul E. McKenney
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index 0cc7f68..dfb048e 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -2926,7 +2926,7 @@ static void __init rcu_boot_init_nocb_percpu_data(struct
rcu_data *rdp)
{
}
-static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
+static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp, int level)
{
return false;
}
[PATCH 12/13] rcu/nocb: Prepare for finegrained deferred wakeup
Provide a way to tune the deferred wakeup level we want to perform from
a safe wakeup point. Currently those sites are:
* nocb_timer
* user/idle/guest entry
* CPU down
* softirq/rcuc
All of these sites perform the wake up for both RCU_NOCB_WAKE and
RCU_NOCB_WAKE_FORCE.
In order to merge nocb_timer and nocb_bypass_timer together, we plan to
add a new RCU_NOCB_WAKE_BYPASS that really want to be deferred until
a timer fires so that we don't wake up the NOCB-gp kthread too early.
To prepare for that, integrate a wake up level/limit that a callsite is
deemed to perform.
Signed-off-by: Frederic Weisbecker
Cc: Josh Triplett
Cc: Lai Jiangshan
Cc: Joel Fernandes
Cc: Neeraj Upadhyay
Cc: Boqun Feng
---
kernel/rcu/tree.c| 2 +-
kernel/rcu/tree.h| 2 +-
kernel/rcu/tree_plugin.h | 15 ---
3 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 2c9cf4df942c..9951a4bef504 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -3823,7 +3823,7 @@ static int rcu_pending(int user)
check_cpu_stall(rdp);
/* Does this CPU need a deferred NOCB wakeup? */
- if (rcu_nocb_need_deferred_wakeup(rdp))
+ if (rcu_nocb_need_deferred_wakeup(rdp, RCU_NOCB_WAKE))
return 1;
/* Is this a nohz_full CPU in userspace or idle? (Ignore RCU if so.) */
diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
index b280a843bd2c..2510e86265c1 100644
--- a/kernel/rcu/tree.h
+++ b/kernel/rcu/tree.h
@@ -433,7 +433,7 @@ static bool rcu_nocb_try_bypass(struct rcu_data *rdp,
struct rcu_head *rhp,
bool *was_alldone, unsigned long flags);
static void __call_rcu_nocb_wake(struct rcu_data *rdp, bool was_empty,
unsigned long flags);
-static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp);
+static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp, int level);
static bool do_nocb_deferred_wakeup(struct rcu_data *rdp);
static void rcu_boot_init_nocb_percpu_data(struct rcu_data *rdp);
static void rcu_spawn_cpu_nocb_kthread(int cpu);
diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h
index d8b50ff40e4b..e0420e3b30e6 100644
--- a/kernel/rcu/tree_plugin.h
+++ b/kernel/rcu/tree_plugin.h
@@ -2364,13 +2364,14 @@ static int rcu_nocb_cb_kthread(void *arg)
}
/* Is a deferred wakeup of rcu_nocb_kthread() required? */
-static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp)
+static int rcu_nocb_need_deferred_wakeup(struct rcu_data *rdp, int level)
{
- return READ_ONCE(rdp->nocb_defer_wakeup) > RCU_NOCB_WAKE_NOT;
+ return READ_ONCE(rdp->nocb_defer_wakeup) >= level;
}
/* Do a deferred wakeup of rcu_nocb_kthread(). */
-static bool do_nocb_deferred_wakeup_common(struct rcu_data *rdp)
+static bool do_nocb_deferred_wakeup_common(struct rcu_data *rdp,
+ int level)
{
unsigned long flags;
int ndw;
@@ -2379,7 +2380,7 @@ static bool do_nocb_deferred_wakeup_common(struct
rcu_data *rdp)
raw_spin_lock_irqsave(_gp->nocb_gp_lock, flags);
- if (!rcu_nocb_need_deferred_wakeup(rdp_gp)) {
+ if (!rcu_nocb_need_deferred_wakeup(rdp_gp, level)) {
raw_spin_unlock_irqrestore(_gp->nocb_gp_lock, flags);
return false;
}
@@ -2396,7 +2397,7 @@ static void do_nocb_deferred_wakeup_timer(struct
timer_list *t)
{
struct rcu_data *rdp = from_timer(rdp, t, nocb_timer);
- do_nocb_deferred_wakeup_common(rdp);
+ do_nocb_deferred_wakeup_common(rdp, RCU_NOCB_WAKE);
}
/*
@@ -2409,8 +2410,8 @@ static bool do_nocb_deferred_wakeup(struct rcu_data *rdp)
if (!rdp->nocb_gp_rdp)
return false;
- if (rcu_nocb_need_deferred_wakeup(rdp->nocb_gp_rdp))
- return do_nocb_deferred_wakeup_common(rdp);
+ if (rcu_nocb_need_deferred_wakeup(rdp->nocb_gp_rdp, RCU_NOCB_WAKE))
+ return do_nocb_deferred_wakeup_common(rdp, RCU_NOCB_WAKE);
return false;
}
--
2.25.1
