On Mon, Dec 24, 2012 at 12:43:25AM +0100, Frederic Weisbecker wrote:
> 2012/12/21 Steven Rostedt <rost...@goodmis.org>:
> > On Thu, 2012-12-20 at 19:32 +0100, Frederic Weisbecker wrote:
> >> Let's imagine you have 4 CPUs. We keep the CPU 0 to offline RCU callbacks
> >> there and to
> >> handle the timekeeping. We set the rest as full dynticks. So you need the
> >> following kernel
> >> parameters:
> >>
> >> rcu_nocbs=1-3 full_nohz=1-3
> >>
> >> (Note rcu_nocbs value must always be the same as full_nohz).
> >
> > Why? You can't have: rcu_nocbs=1-4 full_nohz=1-3
>
> That should be allowed.
>
> > or: rcu_nocbs=1-3 full_nohz=1-4 ?
>
> But that not.
>
> You need to have: rcu_nocbs & full_nohz == full_nohz. This is because
> the tick is not there to maintain the local RCU callbacks anymore. So
> this must be offloaded to the rcu_nocb threads.
>
> I just have a doubt with rcu_nocb. Do we still need the tick to
> complete the grace period for local rcu callbacks? I need to discuss
> that with Paul.
The tick is only needed if rcu_needs_cpu() returns false. Of course,
this means that if you don't invoke rcu_needs_cpu() before returning to
adaptive-idle usermode execution, you are correct that a full_nohz CPU
would also have to be a rcu_nocbs CPU.
That said, I am getting close to having an rcu_needs_cpu() that only
returns false if there are callbacks immediately ready to invoke, at
least if RCU_FAST_NO_HZ=y.
Thanx, Paul
> > That needs to be fixed. Either with a warning, and/or to force the two
> > to be the same. That is, if they specify:
> >
> > rcu_nocbs=1-3 full_nohz=1-4
> >
> > Then set rcu_nocbs=1-4 with a warning about it. Or simply set
> > full_nohz=1-3.
>
> Yep, will do.
>
> Thanks!
>
> >
> > -- Steve
> >
> >>
> >> Now if you want proper isolation you need to:
> >>
> >> * Migrate your processes adequately
> >> * Migrate your irqs to CPU 0
> >> * Migrate the RCU nocb threads to CPU 0. Example with the above
> >> configuration:
> >>
> >> for p in $(ps -o pid= -C rcuo1,rcuo2,rcuo3)
> >> do
> >> taskset -cp 0 $p
> >> done
> >>
> >> Then run what you want on the full dynticks CPUs. For best results, run 1
> >> task
> >> per CPU, mostly in userspace and mostly CPU bound (otherwise more IO =
> >> more kernel
> >> mode execution = more chances to get IPIs, tick restarted, workqueues,
> >> kthreads, etc...)
> >>
> >> This page contains a good reminder for those interested in CPU isolation:
> >> https://github.com/gby/linux/wiki
> >>
> >> But keep in mind that my tree is not yet ready for serious production.
> >>
> >
> >
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
