Re: wmb vs mmiowb

2007-09-03 Thread Nick Piggin
On Thu, Aug 30, 2007 at 02:42:41PM -0500, Brent Casavant wrote: > On Thu, 30 Aug 2007, Nick Piggin wrote: > > > I don't know whether this is exactly a correct implementation of > > Linux's barrier semantics. On one hand, wmb _is_ ordering the stores > > as they come out of the CPU; on the other, i

Re: wmb vs mmiowb

2007-08-30 Thread Brent Casavant
On Thu, 30 Aug 2007, Nick Piggin wrote: > OK, thanks for that. I think I have a rough idea of how they both > work... I was just thinking (hoping) that, although the writel may > not reach the device before the store reaches memory, it would > _appear_ that way from the POV of the device (ie. if t

Re: wmb vs mmiowb

2007-08-29 Thread Nick Piggin
On Wed, Aug 29, 2007 at 01:53:53PM -0500, Brent Casavant wrote: > On Wed, 29 Aug 2007, Nick Piggin wrote: > > > On Tue, Aug 28, 2007 at 03:56:28PM -0500, Brent Casavant wrote: > > > > The simplistic method to solve this is a lock around the section > > > issuing IOs, thereby ensuring serializatio

Re: wmb vs mmiowb

2007-08-29 Thread Brent Casavant
On Wed, 29 Aug 2007, Nick Piggin wrote: > On Tue, Aug 28, 2007 at 03:56:28PM -0500, Brent Casavant wrote: > > The simplistic method to solve this is a lock around the section > > issuing IOs, thereby ensuring serialization of access to the IO > > device. However, as SN2 does not enforce an order

Re: wmb vs mmiowb

2007-08-28 Thread Peter Chubb
> "Brent" == Brent Casavant <[EMAIL PROTECTED]> writes: Brent> That reminds me. Are the people who are working on the Brent> user-level driver effort including a capability similar to Brent> mmiowb()? If we had that capability we could eventually do Brent> away with the change mentioned abov

Re: wmb vs mmiowb

2007-08-28 Thread Brent Casavant
On Thu, 23 Aug 2007, Jesse Barnes wrote: > On Thursday, August 23, 2007 12:27 am Benjamin Herrenschmidt wrote: > > > Of course, the normal memory barrier would usually be a > > > "spin_unlock()" or something like that, not a "wmb()". In fact, I > > > don't think the powerpc implementation (as an e

Re: wmb vs mmiowb

2007-08-28 Thread Brent Casavant
On Fri, 24 Aug 2007, Nick Piggin wrote: > And all platforms other than sn2 don't appear to reorder IOs after > they leave the CPU, so only sn2 needs to do the mmiowb thing before > spin_unlock. I'm sure all of the following is already known to most readers, but I thought the paragraph above might

Re: wmb vs mmiowb

2007-08-28 Thread Nick Piggin
On Tue, Aug 28, 2007 at 03:56:28PM -0500, Brent Casavant wrote: > On Fri, 24 Aug 2007, Nick Piggin wrote: > > > And all platforms other than sn2 don't appear to reorder IOs after > > they leave the CPU, so only sn2 needs to do the mmiowb thing before > > spin_unlock. > > I'm sure all of the follo

Re: wmb vs mmiowb

2007-08-23 Thread Nick Piggin
On Thu, Aug 23, 2007 at 09:56:16AM -0700, Jesse Barnes wrote: > On Thursday, August 23, 2007 12:27 am Benjamin Herrenschmidt wrote: > > > Of course, the normal memory barrier would usually be a > > > "spin_unlock()" or something like that, not a "wmb()". In fact, I > > > don't think the powerpc imp

Re: wmb vs mmiowb

2007-08-23 Thread Nick Piggin
On Thu, Aug 23, 2007 at 06:27:42PM +0200, Benjamin Herrenschmidt wrote: > On Thu, 2007-08-23 at 09:16 -0700, Linus Torvalds wrote: > > > > On Thu, 23 Aug 2007, Nick Piggin wrote: > > > > > > Also, FWIW, there are some advantages of deferring the mmiowb thingy > > > until the point of unlock. > >

Re: wmb vs mmiowb

2007-08-23 Thread Nick Piggin
On Thu, Aug 23, 2007 at 09:16:42AM -0700, Linus Torvalds wrote: > > > On Thu, 23 Aug 2007, Nick Piggin wrote: > > > > Also, FWIW, there are some advantages of deferring the mmiowb thingy > > until the point of unlock. > > And that is exactly what ppc64 does. > > But you're missing a big point:

Re: wmb vs mmiowb

2007-08-23 Thread Jesse Barnes
> > Yeah, they keep threatening to use this instead, but I'm not sure > > how easy it would be. Also they may have more devices/drivers to > > worry about than sn2, so maybe changing over would mean too much > > driver debugging (well auditing really since it's not that hard to > > know where to p

Re: wmb vs mmiowb

2007-08-23 Thread Jesse Barnes
On Thursday, August 23, 2007 12:27 am Benjamin Herrenschmidt wrote: > > Of course, the normal memory barrier would usually be a > > "spin_unlock()" or something like that, not a "wmb()". In fact, I > > don't think the powerpc implementation (as an example of this) will > > actually synchronize with

Re: wmb vs mmiowb

2007-08-23 Thread Benjamin Herrenschmidt
On Thu, 2007-08-23 at 09:16 -0700, Linus Torvalds wrote: > > On Thu, 23 Aug 2007, Nick Piggin wrote: > > > > Also, FWIW, there are some advantages of deferring the mmiowb thingy > > until the point of unlock. > > And that is exactly what ppc64 does. > > But you're missing a big point: for 99.9%

Re: wmb vs mmiowb

2007-08-23 Thread Linus Torvalds
On Thu, 23 Aug 2007, Nick Piggin wrote: > > Also, FWIW, there are some advantages of deferring the mmiowb thingy > until the point of unlock. And that is exactly what ppc64 does. But you're missing a big point: for 99.9% of all hardware, mmiowb() is a total no-op. So when you talk about "adva

Re: wmb vs mmiowb

2007-08-23 Thread Linus Torvalds
On Thu, 23 Aug 2007, Nick Piggin wrote: > > OK, but we'd have some kind of functions that are called not to > serialise the CPUs, but to serialise the IO. It would be up to > the calling code to already provide CPU synchronisation. > > serialize_io(); / unserialize_io(); / a nicer name We coul

Re: wmb vs mmiowb

2007-08-23 Thread Benjamin Herrenschmidt
> Of course, the normal memory barrier would usually be a "spin_unlock()" or > something like that, not a "wmb()". In fact, I don't think the powerpc > implementation (as an example of this) will actually synchronize with > anything *but* a spin_unlock(). We are even more sneaky in the sense t

Re: wmb vs mmiowb

2007-08-23 Thread Benjamin Herrenschmidt
On Wed, 2007-08-22 at 06:57 +0200, Nick Piggin wrote: > It doesn't seem like this primary function of mmiowb has anything to do > with a write barrier that we are used to (it may have a seconary semantic > of a wmb as well, but let's ignore that for now). A write barrier will > never provide you w

Re: wmb vs mmiowb

2007-08-22 Thread Nick Piggin
On Wed, Aug 22, 2007 at 07:57:56PM -0700, Linus Torvalds wrote: > > > On Thu, 23 Aug 2007, Nick Piggin wrote: > > > > > Irix actually had an io_unlock() routine that did this > > > implicitly, but iirc that was shot down for Linux... > > > > Why was it shot down? Seems like a pretty good idea t

Re: wmb vs mmiowb

2007-08-22 Thread Nick Piggin
On Wed, Aug 22, 2007 at 07:57:56PM -0700, Linus Torvalds wrote: > > > On Thu, 23 Aug 2007, Nick Piggin wrote: > > > > > Irix actually had an io_unlock() routine that did this > > > implicitly, but iirc that was shot down for Linux... > > > > Why was it shot down? Seems like a pretty good idea t

Re: wmb vs mmiowb

2007-08-22 Thread Linus Torvalds
On Thu, 23 Aug 2007, Nick Piggin wrote: > > > Irix actually had an io_unlock() routine that did this > > implicitly, but iirc that was shot down for Linux... > > Why was it shot down? Seems like a pretty good idea to me ;) It's horrible. We'd need it for *every* single spinlock type. We have l

Re: wmb vs mmiowb

2007-08-22 Thread Nick Piggin
On Wed, Aug 22, 2007 at 12:02:11PM -0700, Jesse Barnes wrote: > On Wednesday, August 22, 2007 11:07 am Linus Torvalds wrote: > > > It really seems like it is some completely different concept from a > > > barrier. And it shows, on the platform where it really matters > > > (sn2), where the thing ac

Re: wmb vs mmiowb

2007-08-22 Thread Nick Piggin
On Wed, Aug 22, 2007 at 11:07:32AM -0700, Linus Torvalds wrote: > > > On Wed, 22 Aug 2007, Nick Piggin wrote: > > > > It took me more than a glance to see what the difference is supposed to be > > between wmb() and mmiowb(). I think especially because mmiowb isn't really > > like a write barrier

Re: wmb vs mmiowb

2007-08-22 Thread Jesse Barnes
On Wednesday, August 22, 2007 11:07 am Linus Torvalds wrote: > > It really seems like it is some completely different concept from a > > barrier. And it shows, on the platform where it really matters > > (sn2), where the thing actually spins. > > I agree that it probably isn't a "write barrier" per

Re: wmb vs mmiowb

2007-08-22 Thread Linus Torvalds
On Wed, 22 Aug 2007, Nick Piggin wrote: > > It took me more than a glance to see what the difference is supposed to be > between wmb() and mmiowb(). I think especially because mmiowb isn't really > like a write barrier. Well, it is, but it isn't. Not on its own - but together with a "normal" b

wmb vs mmiowb

2007-08-21 Thread Nick Piggin
Hi, I'm ignorant when it comes to IO access, so I hope this isn't rubbish (if it is, I would appreciate being corrected). It took me more than a glance to see what the difference is supposed to be between wmb() and mmiowb(). I think especially because mmiowb isn't really like a write barrier. wm