Re: Preemptable Ticket Spinlock

2013-05-30 Thread Thomas Gleixner
On Thu, 30 May 2013, Raghavendra K T wrote: > Here is the branch with pvpspinlock V9 version in github reabsed to 3.10-rc > > https://github.com/ktraghavendra/linux/tree/pvspinlock_v9 > > planning post a formal email in a separate thread with link a to this > branch (instead of spamming with 19

Re: Preemptable Ticket Spinlock

2013-05-30 Thread Raghavendra K T
On 04/23/2013 07:12 AM, Raghavendra K T wrote: On 04/23/2013 01:19 AM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: On 04/22/2013 07:51 AM, Peter Zijlstra wrote: On Sun, 2013-04-21 at 17:12 -0400, Rik van Riel wrote: If we always incremented the ticket number

Re: Preemptable Ticket Spinlock

2013-04-26 Thread Andrew Theurer
unning on a non paravirtual VMM, and by 7:91X when running on a > VMM that supports a paravirtual locking interface (using a pv > preemptable ticket spinlock), when executing a set of microbenchmarks > as well as a realistic e-commerce benchmark. > > > A detailed algorithm des

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Gleb Natapov
On Mon, Apr 22, 2013 at 07:08:06PM -0400, Rik van Riel wrote: > On 04/22/2013 04:55 PM, Peter Zijlstra wrote: > >On Mon, 2013-04-22 at 16:46 -0400, Jiannan Ouyang wrote: > > >>- pv-preemptable-lock has much less performance variance compare to > >>pv_lock, because it adapts to preemption within V

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Raghavendra K T
On 04/23/2013 02:31 AM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 16:49 -0400, Rik van Riel wrote: Given the fairly high cost of rescheduling a VCPU (which is likely to include an IPI), versus the short hold time of most spinlocks, I have the strong suspicion that your approach would win.

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Raghavendra K T
On 04/22/2013 10:12 PM, Jiannan Ouyang wrote: On Mon, Apr 22, 2013 at 1:58 AM, Raghavendra K T wrote: [...] static __always_inline void __ticket_spin_lock(arch_spinlock_t *lock) { register struct __raw_tickets inc = { .tail = 1 }; + unsigned int timeout = 0; + __t

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Raghavendra K T
On 04/23/2013 01:19 AM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: On 04/22/2013 07:51 AM, Peter Zijlstra wrote: On Sun, 2013-04-21 at 17:12 -0400, Rik van Riel wrote: If we always incremented the ticket number by 2 (instead of 1), then we could use the lower

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 05:56 PM, Andi Kleen wrote: Rik van Riel writes: If we always incremented the ticket number by 2 (instead of 1), then we could use the lower bit of the ticket number as the spinlock. Spinning on a single bit is very inefficient, as you need to do try lock in a loop which is ver

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 04:55 PM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 16:46 -0400, Jiannan Ouyang wrote: - pv-preemptable-lock has much less performance variance compare to pv_lock, because it adapts to preemption within VM, other than using rescheduling that increase VM interference I wou

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Andi Kleen
Rik van Riel writes: > > If we always incremented the ticket number by 2 (instead of 1), then > we could use the lower bit of the ticket number as the spinlock. Spinning on a single bit is very inefficient, as you need to do try lock in a loop which is very unfriendly to the MESI state protocol.

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Jiannan Ouyang
On Mon, Apr 22, 2013 at 4:55 PM, Peter Zijlstra wrote: > > Which pv_lock? The current pv spinlock mess is basically the old unfair > thing. The later patch series I referred to earlier implemented a > paravirt ticket lock, that should perform much better under overcommit. > Yes, it is a paravirt

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 16:49 -0400, Rik van Riel wrote: > Given the fairly high cost of rescheduling a VCPU (which is likely > to include an IPI), versus the short hold time of most spinlocks, > I have the strong suspicion that your approach would win. https://lkml.org/lkml/2012/5/2/101 If you s

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 16:46 -0400, Jiannan Ouyang wrote: > On Mon, Apr 22, 2013 at 4:08 PM, Peter Zijlstra wrote: > > > > > I much prefer the entire series from Jeremy since it maintains the > > ticket semantics and doesn't degrade the lock to unfair under > > contention. > > > > Now I suppose th

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Chegu Vinod
On 4/22/2013 1:50 PM, Jiannan Ouyang wrote: On Mon, Apr 22, 2013 at 4:44 PM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 16:32 -0400, Rik van Riel wrote: IIRC one of the reasons was that the performance improvement wasn't as obvious. Rescheduling VCPUs takes a fair amount of time, quite proba

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 04:48 PM, Peter Zijlstra wrote: Hmm.. it looked like under light overcommit the paravirt ticket lock still had some gain (~10%) and of course it brings the fairness thing which is always good. I can only imagine the mess unfair + vcpu preemption can bring to guest tasks. If you

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Jiannan Ouyang
On Mon, Apr 22, 2013 at 4:44 PM, Peter Zijlstra wrote: > On Mon, 2013-04-22 at 16:32 -0400, Rik van Riel wrote: >> >> IIRC one of the reasons was that the performance improvement wasn't >> as obvious. Rescheduling VCPUs takes a fair amount of time, quite >> probably more than the typical hold tim

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 04:46 PM, Jiannan Ouyang wrote: It would still be very interesting to conduct more experiments to compare these two, to see if the fairness enforced by pv_lock is mandatory, and if preemptable-lock outperforms pv_lock in most cases, and how do they work with PLE. Given the fairly

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 22:44 +0200, Peter Zijlstra wrote: > On Mon, 2013-04-22 at 16:32 -0400, Rik van Riel wrote: > > > > IIRC one of the reasons was that the performance improvement wasn't > > as obvious. Rescheduling VCPUs takes a fair amount of time, quite > > probably more than the typical ho

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Jiannan Ouyang
On Mon, Apr 22, 2013 at 4:08 PM, Peter Zijlstra wrote: > > I much prefer the entire series from Jeremy since it maintains the > ticket semantics and doesn't degrade the lock to unfair under > contention. > > Now I suppose there's a reason its not been merged yet and I suspect > its !paravirt hotp

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 16:32 -0400, Rik van Riel wrote: > > IIRC one of the reasons was that the performance improvement wasn't > as obvious. Rescheduling VCPUs takes a fair amount of time, quite > probably more than the typical hold time of a spinlock. IIRC it would spin for a while before block

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 04:08 PM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 15:56 -0400, Rik van Riel wrote: On 04/22/2013 03:49 PM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: If the native spin_lock code has been called already at that time, the native code would s

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 15:56 -0400, Rik van Riel wrote: > On 04/22/2013 03:49 PM, Peter Zijlstra wrote: > > On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: > > >> If the native spin_lock code has been called already at > >> that time, the native code would still need to be modified > >> to i

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Jiannan Ouyang
On Mon, Apr 22, 2013 at 3:56 PM, Rik van Riel wrote: > On 04/22/2013 03:49 PM, Peter Zijlstra wrote: >> >> On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: > > >>> If the native spin_lock code has been called already at >>> that time, the native code would still need to be modified >>> to in

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 03:49 PM, Peter Zijlstra wrote: On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: If the native spin_lock code has been called already at that time, the native code would still need to be modified to increment the ticket number by 2, so we end up with a compatible value in e

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Mon, 2013-04-22 at 08:52 -0400, Rik van Riel wrote: > On 04/22/2013 07:51 AM, Peter Zijlstra wrote: > > On Sun, 2013-04-21 at 17:12 -0400, Rik van Riel wrote: > >> > >> If we always incremented the ticket number by 2 (instead of 1), then > >> we could use the lower bit of the ticket number as th

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Jiannan Ouyang
On Mon, Apr 22, 2013 at 1:58 AM, Raghavendra K T wrote: > The intuition is to >> >> downgrade a fair lock to an unfair lock automatically upon preemption, >> and preserve the fairness otherwise. > > > I also hope being little unfair, does not affect the original intention > of introducing ticket

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Rik van Riel
On 04/22/2013 07:51 AM, Peter Zijlstra wrote: On Sun, 2013-04-21 at 17:12 -0400, Rik van Riel wrote: If we always incremented the ticket number by 2 (instead of 1), then we could use the lower bit of the ticket number as the spinlock. ISTR that paravirt ticket locks already do that and use th

Re: Preemptable Ticket Spinlock

2013-04-22 Thread Peter Zijlstra
On Sun, 2013-04-21 at 17:12 -0400, Rik van Riel wrote: > > If we always incremented the ticket number by 2 (instead of 1), then > we could use the lower bit of the ticket number as the spinlock. ISTR that paravirt ticket locks already do that and use the lsb to indicate the unlock needs to perfor

Re: Preemptable Ticket Spinlock

2013-04-21 Thread Raghavendra K T
On 04/22/2013 04:37 AM, Jiannan Ouyang wrote: On Sun, Apr 21, 2013 at 5:12 PM, Rik van Riel wrote: Your algorithm is very clever, and very promising. However, it does increase the size of the struct spinlock, and adds an additional atomic operation to spin_unlock, neither of which I suspect ar

Re: Preemptable Ticket Spinlock

2013-04-21 Thread Raghavendra K T
interface (using a pv preemptable ticket spinlock), when executing a set of microbenchmarks as well as a realistic e-commerce benchmark. AFAIU, the experiments are on non PLE machines and it would be worth experimenting on PLE machines too. and also bigger machines. (we may get some surprises there

Re: Preemptable Ticket Spinlock

2013-04-21 Thread Jiannan Ouyang
On Sun, Apr 21, 2013 at 5:12 PM, Rik van Riel wrote: > Your algorithm is very clever, and very promising. > > However, it does increase the size of the struct spinlock, and adds > an additional atomic operation to spin_unlock, neither of which I > suspect are necessary. > > If we always incremente

Re: Preemptable Ticket Spinlock

2013-04-21 Thread Rik van Riel
supports a paravirtual locking interface (using a pv preemptable ticket spinlock), when executing a set of microbenchmarks as well as a realistic e-commerce benchmark. A detailed algorithm description can be found in my VEE 2013 paper, Preemptable Ticket Spinlocks: Improving Consolidated Performance