On Fri, May 15, 2020 at 01:22:31PM +0100, Mel Gorman wrote: > On Fri, May 15, 2020 at 01:28:15PM +0200, Peter Zijlstra wrote:
> > > + if (val & _TIF_POLLING_NRFLAG) > > > + goto activate; > > > > I'm completely confused... the result here is that if you're polling you > > do _NOT_ queue on the wake_list, but instead immediately enqueue. > > > > (which kinda makes sense, since if the remote CPU is idle, it doesn't > > have these lines in its cache anyway) > > > > Crap, I rushed this and severely confused myself about what is going Hehe, and here I though I was confused :-) > on. It is definitely the case that flipping this check does not give > any benefit. The patch shows a benefit but I'm failing to understand > exactly why. How I ended up here was perf indicating a lot of time spent > on smp_cond_load_acquire() which made me look closely at ttwu_remote() > and looking at function graphs to compare the different types of wakeups > and their timings. So the raisin we did this remote wakeup thing in the first place was that Oracle was having very heavy rq->lock cache-line contention. By farming off the enqueue to the CPU that was going to run the task anyway, the rq->lock (and the other runqueue structure lines) could stay in the CPU that was using them (hard). Less cacheline ping-pong, more win. The observation here is that if a CPU is idle, it's rq will not be contended.
