Re: [tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
On Mon, Aug 12, 2019 at 05:52:04AM -0700 tip-bot for Phil Auld wrote:
> Commit-ID: a46d14eca7b75fffe35603aa8b81df654353d80f
> Gitweb:
> https://git.kernel.org/tip/a46d14eca7b75fffe35603aa8b81df654353d80f
> Author: Phil Auld
> AuthorDate: Thu, 1 Aug 2019 09:37:49 -0400
> Committer: Thomas Gleixner
> CommitDate: Mon, 12 Aug 2019 14:45:34 +0200
>
> sched/fair: Use rq_lock/unlock in online_fair_sched_group
>
> Enabling WARN_DOUBLE_CLOCK in /sys/kernel/debug/sched_features causes
> warning to fire in update_rq_clock. This seems to be caused by onlining
> a new fair sched group not using the rq lock wrappers.
>
> [] rq->clock_update_flags & RQCF_UPDATED
> [] WARNING: CPU: 5 PID: 54385 at kernel/sched/core.c:210
> update_rq_clock+0xec/0x150
>
> [] Call Trace:
> [] online_fair_sched_group+0x53/0x100
> [] cpu_cgroup_css_online+0x16/0x20
> [] online_css+0x1c/0x60
> [] cgroup_apply_control_enable+0x231/0x3b0
> [] cgroup_mkdir+0x41b/0x530
> [] kernfs_iop_mkdir+0x61/0xa0
> [] vfs_mkdir+0x108/0x1a0
> [] do_mkdirat+0x77/0xe0
> [] do_syscall_64+0x55/0x1d0
> [] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>
> Using the wrappers in online_fair_sched_group instead of the raw locking
> removes this warning.
>
> [ tglx: Use rq_*lock_irq() ]
>
> Signed-off-by: Phil Auld
> Signed-off-by: Peter Zijlstra (Intel)
> Signed-off-by: Thomas Gleixner
> Cc: Ingo Molnar
> Cc: Vincent Guittot
> Cc: Ingo Molnar
> Link: https://lkml.kernel.org/r/20190801133749.11033-1-pa...@redhat.com
> ---
> kernel/sched/fair.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 19c58599e967..1054d2cf6aaa 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10281,18 +10281,18 @@ err:
> void online_fair_sched_group(struct task_group *tg)
> {
> struct sched_entity *se;
> + struct rq_flags rf;
> struct rq *rq;
> int i;
>
> for_each_possible_cpu(i) {
> rq = cpu_rq(i);
> se = tg->se[i];
> -
> - raw_spin_lock_irq(>lock);
> + rq_lock_irq(rq, );
> update_rq_clock(rq);
> attach_entity_cfs_rq(se);
> sync_throttle(tg, i);
> - raw_spin_unlock_irq(>lock);
> + rq_unlock_irq(rq, );
> }
> }
>
Thanks Thomas!
--
[tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
Commit-ID: a46d14eca7b75fffe35603aa8b81df654353d80f
Gitweb: https://git.kernel.org/tip/a46d14eca7b75fffe35603aa8b81df654353d80f
Author: Phil Auld
AuthorDate: Thu, 1 Aug 2019 09:37:49 -0400
Committer: Thomas Gleixner
CommitDate: Mon, 12 Aug 2019 14:45:34 +0200
sched/fair: Use rq_lock/unlock in online_fair_sched_group
Enabling WARN_DOUBLE_CLOCK in /sys/kernel/debug/sched_features causes
warning to fire in update_rq_clock. This seems to be caused by onlining
a new fair sched group not using the rq lock wrappers.
[] rq->clock_update_flags & RQCF_UPDATED
[] WARNING: CPU: 5 PID: 54385 at kernel/sched/core.c:210
update_rq_clock+0xec/0x150
[] Call Trace:
[] online_fair_sched_group+0x53/0x100
[] cpu_cgroup_css_online+0x16/0x20
[] online_css+0x1c/0x60
[] cgroup_apply_control_enable+0x231/0x3b0
[] cgroup_mkdir+0x41b/0x530
[] kernfs_iop_mkdir+0x61/0xa0
[] vfs_mkdir+0x108/0x1a0
[] do_mkdirat+0x77/0xe0
[] do_syscall_64+0x55/0x1d0
[] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Using the wrappers in online_fair_sched_group instead of the raw locking
removes this warning.
[ tglx: Use rq_*lock_irq() ]
Signed-off-by: Phil Auld
Signed-off-by: Peter Zijlstra (Intel)
Signed-off-by: Thomas Gleixner
Cc: Ingo Molnar
Cc: Vincent Guittot
Cc: Ingo Molnar
Link: https://lkml.kernel.org/r/20190801133749.11033-1-pa...@redhat.com
---
kernel/sched/fair.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 19c58599e967..1054d2cf6aaa 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10281,18 +10281,18 @@ err:
void online_fair_sched_group(struct task_group *tg)
{
struct sched_entity *se;
+ struct rq_flags rf;
struct rq *rq;
int i;
for_each_possible_cpu(i) {
rq = cpu_rq(i);
se = tg->se[i];
-
- raw_spin_lock_irq(>lock);
+ rq_lock_irq(rq, );
update_rq_clock(rq);
attach_entity_cfs_rq(se);
sync_throttle(tg, i);
- raw_spin_unlock_irq(>lock);
+ rq_unlock_irq(rq, );
}
}
Re: [tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
On 8/9/19 7:28 PM, Phil Auld wrote:
> On Fri, Aug 09, 2019 at 06:21:22PM +0200 Dietmar Eggemann wrote:
>> On 8/8/19 1:01 PM, tip-bot for Phil Auld wrote:
[...]
>> Shouldn't this be:
>>
>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
>> index d9407517dae9..1054d2cf6aaa 100644
>> --- a/kernel/sched/fair.c
>> +++ b/kernel/sched/fair.c
>> @@ -10288,11 +10288,11 @@ void online_fair_sched_group(struct task_group
>> *tg)
>> for_each_possible_cpu(i) {
>> rq = cpu_rq(i);
>> se = tg->se[i];
>> - rq_lock(rq, );
>> + rq_lock_irq(rq, );
>> update_rq_clock(rq);
>> attach_entity_cfs_rq(se);
>> sync_throttle(tg, i);
>> - rq_unlock(rq, );
>> + rq_unlock_irq(rq, );
>> }
>> }
>>
>> Currently, you should get a 'inconsistent lock state' warning with
>> CONFIG_PROVE_LOCKING.
>
> Yes, indeed. Sorry about that. Maybe it can be fixed in tip before
> it gets any farther? Or do we need a new patch?
I think Peter is on holiday so maybe you can send a new patch and ask
Ingo or Thomas to replace your original patch on tip sched/core?
Re: [tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
On Fri, Aug 09, 2019 at 06:21:22PM +0200 Dietmar Eggemann wrote:
> On 8/8/19 1:01 PM, tip-bot for Phil Auld wrote:
>
> [...]
>
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 19c58599e967..d9407517dae9 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -10281,18 +10281,18 @@ err:
> > void online_fair_sched_group(struct task_group *tg)
> > {
> > struct sched_entity *se;
> > + struct rq_flags rf;
> > struct rq *rq;
> > int i;
> >
> > for_each_possible_cpu(i) {
> > rq = cpu_rq(i);
> > se = tg->se[i];
> > -
> > - raw_spin_lock_irq(>lock);
> > + rq_lock(rq, );
> > update_rq_clock(rq);
> > attach_entity_cfs_rq(se);
> > sync_throttle(tg, i);
> > - raw_spin_unlock_irq(>lock);
> > + rq_unlock(rq, );
> > }
> > }
>
> Shouldn't this be:
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index d9407517dae9..1054d2cf6aaa 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10288,11 +10288,11 @@ void online_fair_sched_group(struct task_group
> *tg)
> for_each_possible_cpu(i) {
> rq = cpu_rq(i);
> se = tg->se[i];
> - rq_lock(rq, );
> + rq_lock_irq(rq, );
> update_rq_clock(rq);
> attach_entity_cfs_rq(se);
> sync_throttle(tg, i);
> - rq_unlock(rq, );
> + rq_unlock_irq(rq, );
> }
> }
>
> Currently, you should get a 'inconsistent lock state' warning with
> CONFIG_PROVE_LOCKING.
Yes, indeed. Sorry about that. Maybe it can be fixed in tip before
it gets any farther? Or do we need a new patch?
Cheers,
Phil
--
Re: [tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
On 8/8/19 1:01 PM, tip-bot for Phil Auld wrote:
[...]
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 19c58599e967..d9407517dae9 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10281,18 +10281,18 @@ err:
> void online_fair_sched_group(struct task_group *tg)
> {
> struct sched_entity *se;
> + struct rq_flags rf;
> struct rq *rq;
> int i;
>
> for_each_possible_cpu(i) {
> rq = cpu_rq(i);
> se = tg->se[i];
> -
> - raw_spin_lock_irq(>lock);
> + rq_lock(rq, );
> update_rq_clock(rq);
> attach_entity_cfs_rq(se);
> sync_throttle(tg, i);
> - raw_spin_unlock_irq(>lock);
> + rq_unlock(rq, );
> }
> }
Shouldn't this be:
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index d9407517dae9..1054d2cf6aaa 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10288,11 +10288,11 @@ void online_fair_sched_group(struct task_group
*tg)
for_each_possible_cpu(i) {
rq = cpu_rq(i);
se = tg->se[i];
- rq_lock(rq, );
+ rq_lock_irq(rq, );
update_rq_clock(rq);
attach_entity_cfs_rq(se);
sync_throttle(tg, i);
- rq_unlock(rq, );
+ rq_unlock_irq(rq, );
}
}
Currently, you should get a 'inconsistent lock state' warning with
CONFIG_PROVE_LOCKING.
[tip:sched/core] sched/fair: Use rq_lock/unlock in online_fair_sched_group
Commit-ID: 6b8fd01b21f5f2701b407a7118f236ba4c41226d
Gitweb: https://git.kernel.org/tip/6b8fd01b21f5f2701b407a7118f236ba4c41226d
Author: Phil Auld
AuthorDate: Thu, 1 Aug 2019 09:37:49 -0400
Committer: Peter Zijlstra
CommitDate: Thu, 8 Aug 2019 09:09:31 +0200
sched/fair: Use rq_lock/unlock in online_fair_sched_group
Enabling WARN_DOUBLE_CLOCK in /sys/kernel/debug/sched_features causes
warning to fire in update_rq_clock. This seems to be caused by onlining
a new fair sched group not using the rq lock wrappers.
[] rq->clock_update_flags & RQCF_UPDATED
[] WARNING: CPU: 5 PID: 54385 at kernel/sched/core.c:210
update_rq_clock+0xec/0x150
[] Call Trace:
[] online_fair_sched_group+0x53/0x100
[] cpu_cgroup_css_online+0x16/0x20
[] online_css+0x1c/0x60
[] cgroup_apply_control_enable+0x231/0x3b0
[] cgroup_mkdir+0x41b/0x530
[] kernfs_iop_mkdir+0x61/0xa0
[] vfs_mkdir+0x108/0x1a0
[] do_mkdirat+0x77/0xe0
[] do_syscall_64+0x55/0x1d0
[] entry_SYSCALL_64_after_hwframe+0x44/0xa9
Using the wrappers in online_fair_sched_group instead of the raw locking
removes this warning.
Signed-off-by: Phil Auld
Signed-off-by: Peter Zijlstra (Intel)
Cc: Ingo Molnar
Cc: Vincent Guittot
Cc: Ingo Molnar
Link: https://lkml.kernel.org/r/20190801133749.11033-1-pa...@redhat.com
---
kernel/sched/fair.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 19c58599e967..d9407517dae9 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -10281,18 +10281,18 @@ err:
void online_fair_sched_group(struct task_group *tg)
{
struct sched_entity *se;
+ struct rq_flags rf;
struct rq *rq;
int i;
for_each_possible_cpu(i) {
rq = cpu_rq(i);
se = tg->se[i];
-
- raw_spin_lock_irq(>lock);
+ rq_lock(rq, );
update_rq_clock(rq);
attach_entity_cfs_rq(se);
sync_throttle(tg, i);
- raw_spin_unlock_irq(>lock);
+ rq_unlock(rq, );
}
}
