Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-09 Thread Tim Chen
On 4/9/21 8:26 AM, Vincent Guittot wrote: I was expecting idle load balancer to be rate limited to 60 Hz, which >>> >>> Why 60Hz ? >>> >> >> My thinking is we will trigger load balance only after rq->next_balance. >> >> void trigger_load_balance(struct rq *rq) >> { >> /* Don't

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-09 Thread Vincent Guittot
On Fri, 9 Apr 2021 at 01:05, Tim Chen wrote: > > > > > On 4/8/21 7:51 AM, Vincent Guittot wrote: > > >> I was suprised to find the overall cpu% consumption of > >> update_blocked_averages > >> and throughput of the benchmark still didn't change much. So I took a > >> peek into the profile and fo

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-08 Thread Tim Chen
On 4/8/21 7:51 AM, Vincent Guittot wrote: >> I was suprised to find the overall cpu% consumption of >> update_blocked_averages >> and throughput of the benchmark still didn't change much. So I took a >> peek into the profile and found the update_blocked_averages calls shifted to >> the idle

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-08 Thread Vincent Guittot
On Wed, 7 Apr 2021 at 19:19, Tim Chen wrote: > > > > On 4/7/21 7:02 AM, Vincent Guittot wrote: > > Hi Tim, > > > > On Wed, 24 Mar 2021 at 17:05, Tim Chen wrote: > >> > >> > >> > >> On 3/24/21 6:44 AM, Vincent Guittot wrote: > >>> Hi Tim, > >> > >>> > >>> IIUC your problem, we call update_blocked_

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-07 Thread Tim Chen
On 4/7/21 7:02 AM, Vincent Guittot wrote: > Hi Tim, > > On Wed, 24 Mar 2021 at 17:05, Tim Chen wrote: >> >> >> >> On 3/24/21 6:44 AM, Vincent Guittot wrote: >>> Hi Tim, >> >>> >>> IIUC your problem, we call update_blocked_averages() but because of: >>> >>> if (this_rq->avg_idle <

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-04-07 Thread Vincent Guittot
Hi Tim, On Wed, 24 Mar 2021 at 17:05, Tim Chen wrote: > > > > On 3/24/21 6:44 AM, Vincent Guittot wrote: > > Hi Tim, > > > > > IIUC your problem, we call update_blocked_averages() but because of: > > > > if (this_rq->avg_idle < curr_cost + sd->max_newidle_lb_cost) { > >

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-03-24 Thread Tim Chen
On 3/24/21 6:44 AM, Vincent Guittot wrote: > Hi Tim, > > IIUC your problem, we call update_blocked_averages() but because of: > > if (this_rq->avg_idle < curr_cost + sd->max_newidle_lb_cost) { > update_next_balance(sd, &next_balance); >

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-03-24 Thread Vincent Guittot
Hi Tim, Le mardi 23 mars 2021 à 14:37:59 (-0700), Tim Chen a écrit : > > > On 1/29/21 9:27 AM, Vincent Guittot wrote: > > > > The patch below moves the update of the blocked load of CPUs outside > > newidle_balance(). > > On a well known database workload, we also saw a lot of overhead to do

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-03-23 Thread Tim Chen
On 1/29/21 9:27 AM, Vincent Guittot wrote: > > The patch below moves the update of the blocked load of CPUs outside > newidle_balance(). On a well known database workload, we also saw a lot of overhead to do update_blocked_averages in newidle_balance(). So changes to reduce this overhead is

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-04 Thread Qais Yousef
On 02/03/21 18:35, Vincent Guittot wrote: > > > raw_spin_unlock(&this_rq->lock); > > > - /* > > > - * This CPU is going to be idle and blocked load of idle CPUs > > > - * need to be updated. Run the ilb locally as it is a good > > > - * candidate for ilb instead of waking u

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-04 Thread Dietmar Eggemann
On 03/02/2021 14:12, Vincent Guittot wrote: > On Wed, 3 Feb 2021 at 12:54, Dietmar Eggemann > wrote: >> >> On 29/01/2021 18:27, Vincent Guittot wrote: >>> Le vendredi 29 janv. 2021 � 11:33:00 (+0100), Vincent Guittot a �crit : On Thu, 28 Jan 2021 at 16:09, Joel Fernandes wrote: >>

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-03 Thread Joel Fernandes
On Fri, Jan 29, 2021 at 06:27:27PM +0100, Vincent Guittot wrote: [...] > > update_blocked_averages with preempt and irq off is not a good thing > > because we don't manage the number of csf_rq to update and I'm going > > to provide a patchset for this > > The patch below moves the update of the bl

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-03 Thread Vincent Guittot
On Wed, 3 Feb 2021 at 18:09, Qais Yousef wrote: > > On 01/29/21 18:27, Vincent Guittot wrote: > > The patch below moves the update of the blocked load of CPUs outside > > newidle_balance(). > > > > Instead, the update is done with the usual idle load balance update. I'm > > working on an > > add

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-03 Thread Qais Yousef
On 01/29/21 18:27, Vincent Guittot wrote: > The patch below moves the update of the blocked load of CPUs outside > newidle_balance(). > > Instead, the update is done with the usual idle load balance update. I'm > working on an > additonnal patch that will select this cpu that is about to become

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-03 Thread Vincent Guittot
On Wed, 3 Feb 2021 at 12:54, Dietmar Eggemann wrote: > > On 29/01/2021 18:27, Vincent Guittot wrote: > > Le vendredi 29 janv. 2021 � 11:33:00 (+0100), Vincent Guittot a �crit : > >> On Thu, 28 Jan 2021 at 16:09, Joel Fernandes > >> wrote: > >>> > >>> Hi Vincent, > >>> > >>> On Thu, Jan 28, 2

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-03 Thread Dietmar Eggemann
On 29/01/2021 18:27, Vincent Guittot wrote: > Le vendredi 29 janv. 2021 � 11:33:00 (+0100), Vincent Guittot a �crit : >> On Thu, 28 Jan 2021 at 16:09, Joel Fernandes wrote: >>> >>> Hi Vincent, >>> >>> On Thu, Jan 28, 2021 at 8:57 AM Vincent Guittot >>> wrote: > On Mon, Jan 25, 2021 at 03:

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-02-01 Thread Joel Fernandes
On Fri, Jan 29, 2021 at 5:33 AM Vincent Guittot wrote: [...] > > > > > So why is it a problem for you ? You are mentioning newly idle load > > > > > balance so I assume that your root problem is the scheduling delay > > > > > generated by the newly idle load balance which then calls > > > > > upda

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-29 Thread Vincent Guittot
Le vendredi 29 janv. 2021 à 11:33:00 (+0100), Vincent Guittot a écrit : > On Thu, 28 Jan 2021 at 16:09, Joel Fernandes wrote: > > > > Hi Vincent, > > > > On Thu, Jan 28, 2021 at 8:57 AM Vincent Guittot > > wrote: > > > > On Mon, Jan 25, 2021 at 03:42:41PM +0100, Vincent Guittot wrote: > > > > > O

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-28 Thread Qais Yousef
On 01/28/21 10:09, Joel Fernandes wrote: > > > > Qais mentioned half of the time being used by > > > > sugov_next_freq_shared(). Are there any frequency changes resulting in > > > > this call ? > > > > > > I do not see a frequency update happening at the time of the problem. > > > However > > > no

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-28 Thread Joel Fernandes
Hi Vincent, On Thu, Jan 28, 2021 at 8:57 AM Vincent Guittot wrote: > > On Mon, Jan 25, 2021 at 03:42:41PM +0100, Vincent Guittot wrote: > > > On Fri, 22 Jan 2021 at 20:10, Joel Fernandes > > > wrote: > > > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > > > > > On Fri, 22 J

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-28 Thread Vincent Guittot
Hi Joel, On Wed, 27 Jan 2021 at 19:43, Joel Fernandes wrote: > > Hi Vincent, > > On Mon, Jan 25, 2021 at 03:42:41PM +0100, Vincent Guittot wrote: > > On Fri, 22 Jan 2021 at 20:10, Joel Fernandes wrote: > > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > > > > On Fri, 22 Jan

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-27 Thread Joel Fernandes
Hi Vincent, On Mon, Jan 25, 2021 at 03:42:41PM +0100, Vincent Guittot wrote: > On Fri, 22 Jan 2021 at 20:10, Joel Fernandes wrote: > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > > > On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (Google) > > > wrote: > > > > > > > > On an o

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-26 Thread Qais Yousef
On 01/25/21 14:23, Vincent Guittot wrote: > On Fri, 22 Jan 2021 at 19:39, Qais Yousef wrote: > > > > On 01/22/21 17:56, Vincent Guittot wrote: > > > > --- > > > > kernel/sched/fair.c | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/kernel/sched/fair.c b/k

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-25 Thread Vincent Guittot
On Fri, 22 Jan 2021 at 19:39, Qais Yousef wrote: > > On 01/22/21 17:56, Vincent Guittot wrote: > > > --- > > > kernel/sched/fair.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > > index 04a3ce20da67..fe2dc0024db5

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-25 Thread Dietmar Eggemann
On 25/01/2021 18:30, Vincent Guittot wrote: > On Mon, 25 Jan 2021 at 11:45, Dietmar Eggemann > wrote: >> >> On 22/01/2021 20:10, Joel Fernandes wrote: >>> Hi Vincent, >>> >>> Thanks for reply. Please see the replies below: >>> >>> On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: >

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-25 Thread Vincent Guittot
On Mon, 25 Jan 2021 at 11:45, Dietmar Eggemann wrote: > > On 22/01/2021 20:10, Joel Fernandes wrote: > > Hi Vincent, > > > > Thanks for reply. Please see the replies below: > > > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > >> On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (G

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-25 Thread Vincent Guittot
On Fri, 22 Jan 2021 at 20:10, Joel Fernandes wrote: > > Hi Vincent, > > Thanks for reply. Please see the replies below: > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > > On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (Google) > > wrote: > > > > > > On an octacore ARM64 device

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-25 Thread Dietmar Eggemann
On 22/01/2021 20:10, Joel Fernandes wrote: > Hi Vincent, > > Thanks for reply. Please see the replies below: > > On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: >> On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (Google) >> wrote: >>> >>> On an octacore ARM64 device running ChromeO

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-22 Thread Qais Yousef
On 01/22/21 17:56, Vincent Guittot wrote: > > --- > > kernel/sched/fair.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > index 04a3ce20da67..fe2dc0024db5 100644 > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-22 Thread Joel Fernandes
On Fri, Jan 22, 2021 at 06:39:27PM +, Qais Yousef wrote: > On 01/22/21 17:56, Vincent Guittot wrote: > > > --- > > > kernel/sched/fair.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > > > index 04a3ce20da67..fe2d

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-22 Thread Joel Fernandes
Hi Vincent, Thanks for reply. Please see the replies below: On Fri, Jan 22, 2021 at 05:56:22PM +0100, Vincent Guittot wrote: > On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (Google) > wrote: > > > > On an octacore ARM64 device running ChromeOS Linux kernel v5.4, I found > > that there are a lot o

Re: [PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-22 Thread Vincent Guittot
On Fri, 22 Jan 2021 at 16:46, Joel Fernandes (Google) wrote: > > On an octacore ARM64 device running ChromeOS Linux kernel v5.4, I found > that there are a lot of calls to update_blocked_averages(). This causes > the schedule loop to slow down to taking upto 500 micro seconds at > times (due to ne

[PATCH] sched/fair: Rate limit calls to update_blocked_averages() for NOHZ

2021-01-22 Thread Joel Fernandes (Google)
On an octacore ARM64 device running ChromeOS Linux kernel v5.4, I found that there are a lot of calls to update_blocked_averages(). This causes the schedule loop to slow down to taking upto 500 micro seconds at times (due to newidle load balance). I have also seen this manifest in the periodic bala