Benjamin Herrenschmidt <b...@kernel.crashing.org> writes: > On Tue, 2017-04-11 at 14:28 +0200, Cédric Le Goater wrote: >> I really don't know. >> >> Ben, now that we have mttcg activated by default on ppc, it takes >> a while for the linux kernel to do the early setup. I think we are >> in the code section where we spin loop the secondaries. Most of the >> time is spent in skiboot under cpu_idle/relax. >> >> Any idea where that could come from ? >> >> > See c22edfebff29f63d793032e4fbd42a035bb73e27 for an example. >> >> Thanks for the hint. > > They are spinning, but they have smt_low instructions in the loop, > maybe that causes us to do some kind of synchronization as we exit > the emulation loop on these ? I added that to force relinguish time > to other threads on the pre-MT TCG...
Yeah you need a tweak the approach when running with MTTCG as otherwise you end up causing exits for one vCPUs loop to yield to vCPUs that are already running in other threads. > There isn't really such a "pause" instruction. At least not yet.... P9 > has a wait that is meant to wait for special AS_Notify cycles but will > also wait for interrupts. We don't have an mwait at this point. They are worth implementing. FWIW on ARM we only really handle WFI (Wait-for-interrupt) which will cause the EXCP_HALT and that will put the vCPU into a halted state which can be woken up next interrupt. For the other cases YIELD and WFE (wait-for-event) we just treat them as NOPs when MTTCG is enabled (test parallel_cpus). So they will busy-wait spin around the guests wfe code but don't trigger expensive longjmps out of the execution loop. This was all done in: c22edfebff target-arm: don't generate WFE/YIELD calls for MTTCG One other thing I noticed while looking through the PPC stuff is I couldn't see any handling of cpu_reset/powering on. There is a potential race here which ThreadSanitizer will complain about if you start loading up your about-to-be-powered-on-vCPU from another thread. The fix here is to defer the setup with async work. See: 062ba099e0 target-arm/powerctl: defer cpu reset work to CPU context -- Alex Bennée
