On Thu, Jun 03, 2010 at 05:17:30PM +0200, Andi Kleen wrote: > On Thu, Jun 03, 2010 at 10:38:32PM +1000, Nick Piggin wrote: > > And they aren't even using ticket spinlocks!! > > I suppose they simply don't have unfair memory. Makes things easier.
That would certainly be a part of it, I'm sure they have stronger fairness and guarantees at the expense of some performance. We saw the spinlock starvation first on 8-16 core Opterons I think, wheras Altix had been over 1024 cores and POWER7 1024 threads now apparently without reported problems. However I think more is needed than simply "fair" memory at the cache coherency level, considering that for example s390 implements it simply by retrying cas until it succeeds. So you could perfectly round-robin all cache requests for the lock word, but one core could quite easily always find it is granted the cacheline when the lock is already taken. So I think actively enforcing fairness at the lock level would be required. Something like if it is detected that a core is not making progress on a tight cas loop, then it will need to enter a queue of cores where the head of the queue is always granted the cacheline first after it has been dirtied. Interrupts will need to be ignored from this logic. This still doesn't solve the problem of an owner unfairly releasing and grabbing the lock again, they could have more detection to handle that. I don't know how far hardware goes. Maybe it is enough to statistically avoid starvation if memory is pretty fair. But it does seem a lot easier to enforce fairness in software. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
