Well, this definitely have pleased the little toy :) Thank you. I really appreciate your time and effort.
If I may, one more newbie question. What do I need to do for the two patches to find their way into formal kernel code? Thanks -Ofer > -----Original Message----- > From: Peter Zijlstra [mailto:pet...@infradead.org] > Sent: Wednesday, August 9, 2017 6:35 PM > To: Ofer Levi(SW) <ofe...@mellanox.com> > Cc: ru...@rustcorp.com.au; mi...@redhat.com; > vineet.gup...@synopsys.com; linux-kernel@vger.kernel.org; Tejun Heo > <t...@kernel.org> > Subject: Re: hotplug support for arch/arc/plat-eznps platform > > On Wed, Aug 09, 2017 at 03:19:02PM +0000, Ofer Levi(SW) wrote: > > I appreciate your effort and detailed reply, however I'm still > > experiencing performance hit at partition_sched_domains(). It seems the > issue is due to the large magnitude of cpus. > > I used he suggested method 2, patched in the diffs and used the > > command line switch isolcpus to kill load-balancing. > > It did save few hundredth of a sec per cpu. When I limited number of > > available cpus (using present and possible cpus ) to 48, it did reduced > dramatically this function execution time: > > > > With 4K available cpus : > > [ 48.890000] ## CPU16 LIVE ##: Executing Code... > > [ 48.910000] partition_sched_domains start > > [ 49.360000] partition_sched_domains end > > > > With 48 available cpus: > > [ 36.950000] ## CPU16 LIVE ##: Executing Code... > > [ 36.950000] partition_sched_domains start > > [ 36.960000] partition_sched_domains end > > > > Note that I currently use kernel version: 4.8.0.17.0600.00.0000, if this has > any influence. > > Would appreciate your thoughts. > > > > Does something like this cure things? It seems we're doing a possible_cpus > iteration for sysctl cruft, and that will most certainly hurt on your little > toy :-) > > Not sure what the more generic solution to that would be, but the below > avoids it for isolcpus. > > --- > --- a/kernel/sched/core.c > +++ b/kernel/sched/core.c > @@ -85,6 +85,7 @@ int sysctl_sched_rt_runtime = 950000; > > /* CPUs with isolated domains */ > cpumask_var_t cpu_isolated_map; > +cpumask_var_t non_isolated_cpus; > > /* > * __task_rq_lock - lock the rq @p resides on. > @@ -5685,8 +5686,6 @@ static inline void sched_init_smt(void) > > void __init sched_init_smp(void) > { > - cpumask_var_t non_isolated_cpus; > - > alloc_cpumask_var(&non_isolated_cpus, GFP_KERNEL); > > sched_init_numa(); > @@ -5697,17 +5696,17 @@ void __init sched_init_smp(void) > * happen. > */ > mutex_lock(&sched_domains_mutex); > - sched_init_domains(cpu_active_mask); > cpumask_andnot(non_isolated_cpus, cpu_possible_mask, > cpu_isolated_map); > if (cpumask_empty(non_isolated_cpus)) > cpumask_set_cpu(smp_processor_id(), non_isolated_cpus); > + > + sched_init_domains(cpu_active_mask); > mutex_unlock(&sched_domains_mutex); > > /* Move init over to a non-isolated CPU */ > if (set_cpus_allowed_ptr(current, non_isolated_cpus) < 0) > BUG(); > sched_init_granularity(); > - free_cpumask_var(non_isolated_cpus); > > init_sched_rt_class(); > init_sched_dl_class(); > --- a/kernel/sched/debug.c > +++ b/kernel/sched/debug.c > @@ -327,6 +327,8 @@ static struct ctl_table *sd_alloc_ctl_cp > return table; > } > > +extern cpumask_var_t non_isolated_cpus; > + > static struct ctl_table_header *sd_sysctl_header; void > register_sched_domain_sysctl(void) > { > @@ -340,7 +342,7 @@ void register_sched_domain_sysctl(void) > if (entry == NULL) > return; > > - for_each_possible_cpu(i) { > + for_each_cpu(i, non_isolated_cpus) { > snprintf(buf, 32, "cpu%d", i); > entry->procname = kstrdup(buf, GFP_KERNEL); > entry->mode = 0555;
