Re: [PATCH resend ftrace] Asynchronous grace period for register_ftrace_direct()
On Thu, 2 May 2024 16:13:59 -0700 "Paul E. McKenney" wrote: > Very good, and thank you! > > I will drop it from RCU as soon as it shows up in either -next or in > mainline. Sounds good. I'm currently working on updates to get into -rc7 and plan to add my next work on top of that (I know, I know, it's probably the latest release I had for next, but things are still being worked on). -- Steve
Re: [PATCH resend ftrace] Asynchronous grace period for register_ftrace_direct()
On Thu, May 02, 2024 at 05:31:00PM -0400, Steven Rostedt wrote:
> On Wed, 1 May 2024 20:31:06 -0700
> "Paul E. McKenney" wrote:
>
> > On Thu, May 02, 2024 at 11:05:01AM +0900, Masami Hiramatsu wrote:
> > > On Wed, 1 May 2024 16:12:37 -0700
> > > "Paul E. McKenney" wrote:
> > >
> > > > Note that the immediate pressure for this patch should be relieved by
> > > > the
> > > > NAPI patch series [1], but this sort of problem could easily arise
> > > > again.
> > > >
> > > > When running heavy test workloads with KASAN enabled, RCU Tasks grace
> > > > periods can extend for many tens of seconds, significantly slowing
> > > > trace registration. Therefore, make the registration-side RCU Tasks
> > > > grace period be asynchronous via call_rcu_tasks().
> > >
> > > Good catch! AFAICS, there is no reason to wait for synchronization
> > > when adding a new direct trampoline.
> > > This looks good to me.
> > >
> > > Reviewed-by: Masami Hiramatsu (Google)
> >
> > Thank you very much! I will apply this on my next rebase.
>
> I can take it.
>
> It's not a bug fix but just an performance improvement, so it can go into
> the next merge window.
Very good, and thank you!
I will drop it from RCU as soon as it shows up in either -next or in
mainline.
Thanx, Paul
> -- Steve
>
>
>
> >
> > > Thank you,
> > >
> > > > [1]
> > > > https://lore.kernel.org/all/cover.1710877680.git@cloudflare.com/
> > > >
> > > > Reported-by: Jakub Kicinski
> > > > Reported-by: Alexei Starovoitov
> > > > Reported-by: Chris Mason
> > > > Signed-off-by: Paul E. McKenney
> > > > Cc: Steven Rostedt
> > > > Cc: Masami Hiramatsu
> > > > Cc: Mark Rutland
> > > > Cc: Mathieu Desnoyers
> > > > Cc:
> > > >
> > > > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > > > index 6c96b30f3d63b..32ea92934268c 100644
> > > > --- a/kernel/trace/ftrace.c
> > > > +++ b/kernel/trace/ftrace.c
> > > > @@ -5365,6 +5365,13 @@ static void
> > > > remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long }
> > > > }
> > > >
> > > > +static void register_ftrace_direct_cb(struct rcu_head *rhp)
> > > > +{
> > > > + struct ftrace_hash *fhp = container_of(rhp, struct
> > > > ftrace_hash, rcu); +
> > > > + free_ftrace_hash(fhp);
> > > > +}
> > > > +
> > > > /**
> > > > * register_ftrace_direct - Call a custom trampoline directly
> > > > * for multiple functions registered in @ops
> > > > @@ -5463,10 +5470,8 @@ int register_ftrace_direct(struct ftrace_ops
> > > > *ops, unsigned long addr) out_unlock:
> > > > mutex_unlock(_mutex);
> > > >
> > > > - if (free_hash && free_hash != EMPTY_HASH) {
> > > > - synchronize_rcu_tasks();
> > > > - free_ftrace_hash(free_hash);
> > > > - }
> > > > + if (free_hash && free_hash != EMPTY_HASH)
> > > > + call_rcu_tasks(_hash->rcu,
> > > > register_ftrace_direct_cb);
> > > > if (new_hash)
> > > > free_ftrace_hash(new_hash);
> > >
> > >
> > > --
> > > Masami Hiramatsu (Google)
>
Re: [PATCH resend ftrace] Asynchronous grace period for register_ftrace_direct()
On Wed, 1 May 2024 20:31:06 -0700
"Paul E. McKenney" wrote:
> On Thu, May 02, 2024 at 11:05:01AM +0900, Masami Hiramatsu wrote:
> > On Wed, 1 May 2024 16:12:37 -0700
> > "Paul E. McKenney" wrote:
> >
> > > Note that the immediate pressure for this patch should be relieved by the
> > > NAPI patch series [1], but this sort of problem could easily arise again.
> > >
> > > When running heavy test workloads with KASAN enabled, RCU Tasks grace
> > > periods can extend for many tens of seconds, significantly slowing
> > > trace registration. Therefore, make the registration-side RCU Tasks
> > > grace period be asynchronous via call_rcu_tasks().
> >
> > Good catch! AFAICS, there is no reason to wait for synchronization
> > when adding a new direct trampoline.
> > This looks good to me.
> >
> > Reviewed-by: Masami Hiramatsu (Google)
>
> Thank you very much! I will apply this on my next rebase.
I can take it.
It's not a bug fix but just an performance improvement, so it can go into
the next merge window.
-- Steve
>
> > Thank you,
> >
> > > [1]
> > > https://lore.kernel.org/all/cover.1710877680.git@cloudflare.com/
> > >
> > > Reported-by: Jakub Kicinski
> > > Reported-by: Alexei Starovoitov
> > > Reported-by: Chris Mason
> > > Signed-off-by: Paul E. McKenney
> > > Cc: Steven Rostedt
> > > Cc: Masami Hiramatsu
> > > Cc: Mark Rutland
> > > Cc: Mathieu Desnoyers
> > > Cc:
> > >
> > > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > > index 6c96b30f3d63b..32ea92934268c 100644
> > > --- a/kernel/trace/ftrace.c
> > > +++ b/kernel/trace/ftrace.c
> > > @@ -5365,6 +5365,13 @@ static void
> > > remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long }
> > > }
> > >
> > > +static void register_ftrace_direct_cb(struct rcu_head *rhp)
> > > +{
> > > + struct ftrace_hash *fhp = container_of(rhp, struct
> > > ftrace_hash, rcu); +
> > > + free_ftrace_hash(fhp);
> > > +}
> > > +
> > > /**
> > > * register_ftrace_direct - Call a custom trampoline directly
> > > * for multiple functions registered in @ops
> > > @@ -5463,10 +5470,8 @@ int register_ftrace_direct(struct ftrace_ops
> > > *ops, unsigned long addr) out_unlock:
> > > mutex_unlock(_mutex);
> > >
> > > - if (free_hash && free_hash != EMPTY_HASH) {
> > > - synchronize_rcu_tasks();
> > > - free_ftrace_hash(free_hash);
> > > - }
> > > + if (free_hash && free_hash != EMPTY_HASH)
> > > + call_rcu_tasks(_hash->rcu,
> > > register_ftrace_direct_cb);
> > > if (new_hash)
> > > free_ftrace_hash(new_hash);
> >
> >
> > --
> > Masami Hiramatsu (Google)
Re: [PATCH resend ftrace] Asynchronous grace period for register_ftrace_direct()
On Thu, May 02, 2024 at 11:05:01AM +0900, Masami Hiramatsu wrote:
> On Wed, 1 May 2024 16:12:37 -0700
> "Paul E. McKenney" wrote:
>
> > Note that the immediate pressure for this patch should be relieved by the
> > NAPI patch series [1], but this sort of problem could easily arise again.
> >
> > When running heavy test workloads with KASAN enabled, RCU Tasks grace
> > periods can extend for many tens of seconds, significantly slowing
> > trace registration. Therefore, make the registration-side RCU Tasks
> > grace period be asynchronous via call_rcu_tasks().
>
> Good catch! AFAICS, there is no reason to wait for synchronization
> when adding a new direct trampoline.
> This looks good to me.
>
> Reviewed-by: Masami Hiramatsu (Google)
Thank you very much! I will apply this on my next rebase.
Thanx, Paul
> Thank you,
>
> > [1] https://lore.kernel.org/all/cover.1710877680.git@cloudflare.com/
> >
> > Reported-by: Jakub Kicinski
> > Reported-by: Alexei Starovoitov
> > Reported-by: Chris Mason
> > Signed-off-by: Paul E. McKenney
> > Cc: Steven Rostedt
> > Cc: Masami Hiramatsu
> > Cc: Mark Rutland
> > Cc: Mathieu Desnoyers
> > Cc:
> >
> > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> > index 6c96b30f3d63b..32ea92934268c 100644
> > --- a/kernel/trace/ftrace.c
> > +++ b/kernel/trace/ftrace.c
> > @@ -5365,6 +5365,13 @@ static void remove_direct_functions_hash(struct
> > ftrace_hash *hash, unsigned long
> > }
> > }
> >
> > +static void register_ftrace_direct_cb(struct rcu_head *rhp)
> > +{
> > + struct ftrace_hash *fhp = container_of(rhp, struct ftrace_hash, rcu);
> > +
> > + free_ftrace_hash(fhp);
> > +}
> > +
> > /**
> > * register_ftrace_direct - Call a custom trampoline directly
> > * for multiple functions registered in @ops
> > @@ -5463,10 +5470,8 @@ int register_ftrace_direct(struct ftrace_ops *ops,
> > unsigned long addr)
> > out_unlock:
> > mutex_unlock(_mutex);
> >
> > - if (free_hash && free_hash != EMPTY_HASH) {
> > - synchronize_rcu_tasks();
> > - free_ftrace_hash(free_hash);
> > - }
> > + if (free_hash && free_hash != EMPTY_HASH)
> > + call_rcu_tasks(_hash->rcu, register_ftrace_direct_cb);
> >
> > if (new_hash)
> > free_ftrace_hash(new_hash);
>
>
> --
> Masami Hiramatsu (Google)
Re: [PATCH resend ftrace] Asynchronous grace period for register_ftrace_direct()
On Wed, 1 May 2024 16:12:37 -0700
"Paul E. McKenney" wrote:
> Note that the immediate pressure for this patch should be relieved by the
> NAPI patch series [1], but this sort of problem could easily arise again.
>
> When running heavy test workloads with KASAN enabled, RCU Tasks grace
> periods can extend for many tens of seconds, significantly slowing
> trace registration. Therefore, make the registration-side RCU Tasks
> grace period be asynchronous via call_rcu_tasks().
>
Good catch! AFAICS, there is no reason to wait for synchronization
when adding a new direct trampoline.
This looks good to me.
Reviewed-by: Masami Hiramatsu (Google)
Thank you,
> [1] https://lore.kernel.org/all/cover.1710877680.git@cloudflare.com/
>
> Reported-by: Jakub Kicinski
> Reported-by: Alexei Starovoitov
> Reported-by: Chris Mason
> Signed-off-by: Paul E. McKenney
> Cc: Steven Rostedt
> Cc: Masami Hiramatsu
> Cc: Mark Rutland
> Cc: Mathieu Desnoyers
> Cc:
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 6c96b30f3d63b..32ea92934268c 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -5365,6 +5365,13 @@ static void remove_direct_functions_hash(struct
> ftrace_hash *hash, unsigned long
> }
> }
>
> +static void register_ftrace_direct_cb(struct rcu_head *rhp)
> +{
> + struct ftrace_hash *fhp = container_of(rhp, struct ftrace_hash, rcu);
> +
> + free_ftrace_hash(fhp);
> +}
> +
> /**
> * register_ftrace_direct - Call a custom trampoline directly
> * for multiple functions registered in @ops
> @@ -5463,10 +5470,8 @@ int register_ftrace_direct(struct ftrace_ops *ops,
> unsigned long addr)
> out_unlock:
> mutex_unlock(_mutex);
>
> - if (free_hash && free_hash != EMPTY_HASH) {
> - synchronize_rcu_tasks();
> - free_ftrace_hash(free_hash);
> - }
> + if (free_hash && free_hash != EMPTY_HASH)
> + call_rcu_tasks(_hash->rcu, register_ftrace_direct_cb);
>
> if (new_hash)
> free_ftrace_hash(new_hash);
--
Masami Hiramatsu (Google)
