Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-17 Thread Peter Zijlstra
On Tue, May 17, 2016 at 04:22:04PM +0530, Srikar Dronamraju wrote: > > > > Maybe even something like so; which would make Power <= 6 use the > > default topology and result in a shared LLC between the on package cores > > etc.. > > > > Power7 is then special for not having a shared L3 but having

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-17 Thread Srikar Dronamraju
> > Maybe even something like so; which would make Power <= 6 use the > default topology and result in a shared LLC between the on package cores > etc.. > > Power7 is then special for not having a shared L3 but having the > asymmetric SMT thing and Power8 again gains the shared L3 while > retaini

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-17 Thread Peter Zijlstra
On Mon, May 16, 2016 at 04:00:32PM +0200, Peter Zijlstra wrote: > On Fri, May 13, 2016 at 10:12:26AM +1000, Michael Neuling wrote: > > > > Basically; and if so, if its cheap enough to shoot a task to an idle > > > core to avoid queueing. Assuming there still is some cache residency on > > > the ol

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-16 Thread Dietmar Eggemann
On 16/05/16 18:02, Peter Zijlstra wrote: > On Mon, May 16, 2016 at 04:31:08PM +0100, Dietmar Eggemann wrote: >> On 09/05/16 11:48, Peter Zijlstra wrote: >> >> Couldn't you just always access sd->shared via >> sd = rcu_dereference(per_cpu(sd_llc, cpu)) for >> updating nr_busy_cpus? > > Sure; but wh

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-16 Thread Peter Zijlstra
On Mon, May 16, 2016 at 04:31:08PM +0100, Dietmar Eggemann wrote: > On 09/05/16 11:48, Peter Zijlstra wrote: > > Couldn't you just always access sd->shared via > sd = rcu_dereference(per_cpu(sd_llc, cpu)) for > updating nr_busy_cpus? Sure; but why would I want to add that extra dereference? Note

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-16 Thread Dietmar Eggemann
On 09/05/16 11:48, Peter Zijlstra wrote: Couldn't you just always access sd->shared via sd = rcu_dereference(per_cpu(sd_llc, cpu)) for updating nr_busy_cpus? The call_rcu() thing is on the sd any way. @@ -5879,7 +5879,6 @@ static void destroy_sched_domains(struct sched_domain *sd) DEFINE_PER_CP

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-16 Thread Peter Zijlstra
On Fri, May 13, 2016 at 10:12:26AM +1000, Michael Neuling wrote: > > Basically; and if so, if its cheap enough to shoot a task to an idle > > core to avoid queueing. Assuming there still is some cache residency on > > the old core, the inter-core fill should be much cheaper than fetching > > it of

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-12 Thread Michael Neuling
On Thu, 2016-05-12 at 13:33 +0200, Peter Zijlstra wrote: > On Thu, May 12, 2016 at 09:07:52PM +1000, Michael Neuling wrote: > > > > On Thu, 2016-05-12 at 07:07 +0200, Peter Zijlstra wrote: > > > > > > > > But as per the above, Power7 and Power8 have explicit logic to share > > > the > > > per-co

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-12 Thread Peter Zijlstra
On Thu, May 12, 2016 at 09:07:52PM +1000, Michael Neuling wrote: > On Thu, 2016-05-12 at 07:07 +0200, Peter Zijlstra wrote: > > But as per the above, Power7 and Power8 have explicit logic to share the > > per-core L3 with the other cores. > > > > How effective is that? From some of the slides/doc

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-12 Thread Michael Neuling
On Thu, 2016-05-12 at 07:07 +0200, Peter Zijlstra wrote: > On Thu, May 12, 2016 at 12:05:37PM +1000, Michael Neuling wrote: > > > > On Wed, 2016-05-11 at 20:24 +0200, Peter Zijlstra wrote: > > > > > > On Wed, May 11, 2016 at 02:33:45PM +0200, Peter Zijlstra wrote: > > > > > > > > > > > > Hmm, P

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Peter Zijlstra
On Thu, May 12, 2016 at 12:05:37PM +1000, Michael Neuling wrote: > On Wed, 2016-05-11 at 20:24 +0200, Peter Zijlstra wrote: > > On Wed, May 11, 2016 at 02:33:45PM +0200, Peter Zijlstra wrote: > > > > > > Hmm, PPC folks; what does your topology look like? > > > > > > Currently your sched_domain_to

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Michael Neuling
On Wed, 2016-05-11 at 20:24 +0200, Peter Zijlstra wrote: > On Wed, May 11, 2016 at 02:33:45PM +0200, Peter Zijlstra wrote: > > > > Hmm, PPC folks; what does your topology look like? > > > > Currently your sched_domain_topology, as per arch/powerpc/kernel/smp.c > > seems to suggest your cores do n

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Peter Zijlstra
On Wed, May 11, 2016 at 02:33:45PM +0200, Peter Zijlstra wrote: > Hmm, PPC folks; what does your topology look like? > > Currently your sched_domain_topology, as per arch/powerpc/kernel/smp.c > seems to suggest your cores do not share cache at all. > > https://en.wikipedia.org/wiki/POWER7 seems t

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Peter Zijlstra
On Wed, May 11, 2016 at 02:33:45PM +0200, Peter Zijlstra wrote: > Do we want to model this co-operative L3 slices thing as a sort of > node-wide LLC for the purpose of the scheduler ? > > the scheduler does try and scale things by 'assuming' LLC := node. So this whole series is about selecting id

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Matt Fleming
On Wed, 11 May, at 07:37:52PM, Peter Zijlstra wrote: > On Wed, May 11, 2016 at 12:55:56PM +0100, Matt Fleming wrote: > > > This breaks my POWER7 box which presumably doesn't have > > SD_SHARE_PKG_RESOURCES, > > > index 978b3ef2d87e..d27153adee4d 100644 > > --- a/kernel/sched/fair.c > > +++ b/ker

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Peter Zijlstra
On Wed, May 11, 2016 at 12:55:56PM +0100, Matt Fleming wrote: > This breaks my POWER7 box which presumably doesn't have > SD_SHARE_PKG_RESOURCES, > index 978b3ef2d87e..d27153adee4d 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -7920,7 +7920,8 @@ static inline void set_cpu_s

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Peter Zijlstra
On Wed, May 11, 2016 at 12:55:56PM +0100, Matt Fleming wrote: > > --- a/kernel/sched/fair.c > > +++ b/kernel/sched/fair.c > > @@ -7842,13 +7842,13 @@ static inline void set_cpu_sd_state_busy > > int cpu = smp_processor_id(); > > > > rcu_read_lock(); > > - sd = rcu_dereference(per_cpu(sd

Re: [RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-11 Thread Matt Fleming
On Mon, 09 May, at 12:48:11PM, Peter Zijlstra wrote: > Move the nr_busy_cpus thing from its hacky sd->parent->groups->sgc > location into the much more natural sched_domain_shared location. > > Signed-off-by: Peter Zijlstra (Intel) > --- > include/linux/sched.h|1 + > kernel/sched/core.c

[RFC][PATCH 4/7] sched: Replace sd_busy/nr_busy_cpus with sched_domain_shared

2016-05-09 Thread Peter Zijlstra
Move the nr_busy_cpus thing from its hacky sd->parent->groups->sgc location into the much more natural sched_domain_shared location. Signed-off-by: Peter Zijlstra (Intel) --- include/linux/sched.h|1 + kernel/sched/core.c | 10 +- kernel/sched/fair.c | 22 ++