Hi Dietmar, On Tue, May 12, 2020 at 11:00:16AM +0200 Dietmar Eggemann wrote: > On 11/05/2020 22:44, Phil Auld wrote: > > On Mon, May 11, 2020 at 09:25:43PM +0200 Vincent Guittot wrote: > >> On Thu, 7 May 2020 at 22:36, Phil Auld <pa...@redhat.com> wrote: > >>> > >>> sched/fair: Fix enqueue_task_fair warning some more > >>> > >>> The recent patch, fe61468b2cb (sched/fair: Fix enqueue_task_fair warning) > >>> did not fully resolve the issues with the rq->tmp_alone_branch != > >>> &rq->leaf_cfs_rq_list warning in enqueue_task_fair. There is a case where > >>> the first for_each_sched_entity loop exits due to on_rq, having > >>> incompletely > >>> updated the list. In this case the second for_each_sched_entity loop can > >>> further modify se. The later code to fix up the list management fails to > >>> do > >>> what is needed because se no longer points to the sched_entity which broke > >>> out of the first loop. > >>> > >>> Address this by calling leaf_add_rq_list if there are throttled parents > >>> while > >>> doing the second for_each_sched_entity loop. > >>> > >> > >> Fixes: fe61468b2cb (sched/fair: Fix enqueue_task_fair warning) > >> > >>> Suggested-by: Vincent Guittot <vincent.guit...@linaro.org> > >>> Signed-off-by: Phil Auld <pa...@redhat.com> > >>> Cc: Peter Zijlstra (Intel) <pet...@infradead.org> > >>> Cc: Vincent Guittot <vincent.guit...@linaro.org> > >>> Cc: Ingo Molnar <mi...@kernel.org> > >>> Cc: Juri Lelli <juri.le...@redhat.com> > >> > >> With the Fixes tag and the typo mentioned by Tao > >> > > > > Right, that last line of the commit message should read > > "list_add_leaf_cfs_rq" > > > > > >> Reviewed-by: Vincent Guittot <vincent.guit...@linaro.org> > > > > Thanks Vincent. > > > > Peter/Ingo, do you want me to resend or can you fix when applying? > > > Maybe you could add that 'the throttled parent was already added back to > the list by a task enqueue in a parallel child hierarchy'. > > IMHO, this is part of the description because otherwise the throttled > parent would have connected the branch. > > And the not-adding of the intermediate child cfs_rq would have gone > unnoticed.
Okay, I'll add that statement. For those curious here are the lines from about 70ms earlier in the trace where the throttled parent (0xffffa085e48ce000) is added to the list. bz1738415-test-6264 [005] 1271.315046: sched_waking: comm=bz1738415-test pid=6269 prio=120 target_cpu=005 bz1738415-test-6264 [005] 1271.315048: sched_migrate_task: comm=bz1738415-test pid=6269 prio=120 orig_cpu=5 dest_cpu=17 bz1738415-test-6264 [005] 1271.315050: bprint: enqueue_task_fair: se 0xffffa081e6d7de80 on_rq 0 cfs_rq = 0xffffa085e48ce000 bz1738415-test-6264 [005] 1271.315051: bprint: enqueue_entity: Add_leaf_rq: cpu 17: nr_r 2; cfs 0xffffa085e48ce000 onlist 0 tmp_a_b = 0xffffa085ef92c868 &rq->l_c_r_l = 0xffffa085ef92c868 bz1738415-test-6264 [005] 1271.315053: bprint: enqueue_entity: Add_leaf_rq: cpu 17: nr_r 2: parent onlist Set tmp_alone_branch to 0xffffa085ef92c868 bz1738415-test-6264 [005] 1271.315053: bprint: enqueue_task_fair: current se = 0xffffa081e6d7de80, orig_se = 0xffffa081e6d7de80 bz1738415-test-6264 [005] 1271.315055: bprint: enqueue_task_fair: Add_leaf_rq: cpu 17: nr_r 2; cfs 0xffffa085e48ce000 onlist 1 tmp_a_b = 0xffffa085ef92c868 &rq->l_c_r_l = 0xffffa085ef92c868 bz1738415-test-6264 [005] 1271.315056: sched_wake_idle_without_ipi: cpu=17 > > Reviewed-by: Dietmar Eggemann <dietmar.eggem...@arm.com> Thanks, Phil > > [...] > --