Note that the immediate pressure for this patch should be relieved by the NAPI patch series [1], but this sort of problem could easily arise again.
When running heavy test workloads with KASAN enabled, RCU Tasks grace periods can extend for many tens of seconds, significantly slowing trace registration. Therefore, make the registration-side RCU Tasks grace period be asynchronous via call_rcu_tasks(). [1] https://lore.kernel.org/all/cover.1710877680.git....@cloudflare.com/ Reported-by: Jakub Kicinski <k...@kernel.org> Reported-by: Alexei Starovoitov <a...@kernel.org> Reported-by: Chris Mason <c...@fb.com> Signed-off-by: Paul E. McKenney <paul...@kernel.org> Cc: Steven Rostedt <rost...@goodmis.org> Cc: Masami Hiramatsu <mhira...@kernel.org> Cc: Mark Rutland <mark.rutl...@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoy...@efficios.com> Cc: <linux-trace-ker...@vger.kernel.org> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 6c96b30f3d63b..32ea92934268c 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -5365,6 +5365,13 @@ static void remove_direct_functions_hash(struct ftrace_hash *hash, unsigned long } } +static void register_ftrace_direct_cb(struct rcu_head *rhp) +{ + struct ftrace_hash *fhp = container_of(rhp, struct ftrace_hash, rcu); + + free_ftrace_hash(fhp); +} + /** * register_ftrace_direct - Call a custom trampoline directly * for multiple functions registered in @ops @@ -5463,10 +5470,8 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) out_unlock: mutex_unlock(&direct_mutex); - if (free_hash && free_hash != EMPTY_HASH) { - synchronize_rcu_tasks(); - free_ftrace_hash(free_hash); - } + if (free_hash && free_hash != EMPTY_HASH) + call_rcu_tasks(&free_hash->rcu, register_ftrace_direct_cb); if (new_hash) free_ftrace_hash(new_hash);
