Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread James Bottomley
On Wed, 2007-11-21 at 09:39 +1100, Benjamin Herrenschmidt wrote: > On Tue, 2007-11-20 at 15:10 -0600, James Bottomley wrote: > > We're talking about trying to fix this for 2.4; which is already at > > -rc3 ... Is an entire arch change for dma alignment really a merge > > candidate at this stage? >

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread Benjamin Herrenschmidt
On Tue, 2007-11-20 at 15:10 -0600, James Bottomley wrote: > We're talking about trying to fix this for 2.4; which is already at > -rc3 ... Is an entire arch change for dma alignment really a merge > candidate at this stage? Well, as I said before... it's a matter of what seems to be the less like

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread James Bottomley
On Tue, 2007-11-20 at 12:05 -0800, Roland Dreier wrote: > > Actually, we already established on IRC that the lasi700 driver doesn't > > need this, principally because the parisc architecture doesn't do an > > invalidate for DMA_FROM_DEVICE but a flush and invalidate > > (architecturally, if you

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread Roland Dreier
> Actually, we already established on IRC that the lasi700 driver doesn't > need this, principally because the parisc architecture doesn't do an > invalidate for DMA_FROM_DEVICE but a flush and invalidate > (architecturally, if you read our manuals, even pdc is entitled to write > back dirty l

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread James Bottomley
On Tue, 2007-11-20 at 14:14 +1100, Benjamin Herrenschmidt wrote: > FYI, Here's what I have for the SCSI change. I haven't updated drivers > to care for the new return code though, help appreciated with that as I > don't know much about these drivers. It looks to me like the return problem could b

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread James Bottomley
On Tue, 2007-11-20 at 09:29 +0100, Thomas Bogendoerfer wrote: > On Tue, Nov 20, 2007 at 06:51:14AM +1100, Benjamin Herrenschmidt wrote: > > > > On Mon, 2007-11-19 at 00:38 -0800, David Miller wrote: > > > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > > > Date: Mon, 19 Nov 2007 16:35:23 +1100

Re: SCSI breakage on non-cache coherent architectures

2007-11-20 Thread Thomas Bogendoerfer
On Tue, Nov 20, 2007 at 06:51:14AM +1100, Benjamin Herrenschmidt wrote: > > On Mon, 2007-11-19 at 00:38 -0800, David Miller wrote: > > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > > Date: Mon, 19 Nov 2007 16:35:23 +1100 > > > > > I'm not sure what is the best way to fix that. Internally, I'

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
FYI, Here's what I have for the SCSI change. I haven't updated drivers to care for the new return code though, help appreciated with that as I don't know much about these drivers. Index: linux-work/drivers/scsi/scsi_error.c === --- li

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 18:10 -0800, Roland Dreier wrote: > > I wrapped this ugliness up inside the macro back in what I posted in > 2002 (http://lkml.org/lkml/2002/6/12/234): > > #define __dma_buffer __dma_buffer_line(__LINE__) > #define __dma_buffer_line(line) __dma_buffer_expand_line(line) > #d

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Roland Dreier
> 2) Add the __dma_cacheline_aligned tag. > > But note that with #2 it could get quite ugly because the > alignment and size both have a minimum that needs to be > enforced, not just the alignment alone. So either: > > struct foo { > unsigned int other_unrelated_stuff; > > s

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread David Miller
From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> Date: Tue, 20 Nov 2007 11:55:01 +1100 > BTW. What is the status nowadays with skb's ? Good question. Some drivers are problematic (or were) because they put DMA descriptor chaining information at the head of the buffer, but those have been fixed e

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 16:46 -0800, David Miller wrote: > > 1) Require that entire buffers are commited by call sites, >and thus "embedding" DMA'd within non-DMA stuff isn't allowed > > 2) Add the __dma_cacheline_aligned tag. > > But note that with #2 it could get quite ugly because the > al

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread David Miller
From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> Date: Tue, 20 Nov 2007 11:34:24 +1100 > Do you still think we should introduce this __dma_cacheline_aligned ? Do > you see other cases of drivers where it would be useful ? It tend to > agree with your earlier statement that drivers doing that are b

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 14:31 -0800, David Miller wrote: > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > Date: Tue, 20 Nov 2007 06:51:14 +1100 > > > On Mon, 2007-11-19 at 00:38 -0800, David Miller wrote: > > > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > > > Date: Mon, 19 Nov 2007 16:35:

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread David Miller
From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> Date: Tue, 20 Nov 2007 06:51:14 +1100 > On Mon, 2007-11-19 at 00:38 -0800, David Miller wrote: > > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > > Date: Mon, 19 Nov 2007 16:35:23 +1100 > > > > You could make a dma_cacheline_aligned and use

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 13:43 -0800, Roland Dreier wrote: > > I've been debugging various issues on the PowerPC 44x embedded > > architecture which happens to have non-coherent PCI DMA. > > > > One of the problem I'm hitting is that one really need to enforce > > kmalloc alignement to cache lin

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Roland Dreier
> I've been debugging various issues on the PowerPC 44x embedded > architecture which happens to have non-coherent PCI DMA. > > One of the problem I'm hitting is that one really need to enforce > kmalloc alignement to cache lines or bad things will happen (among > others with USB), for some

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
> I'd like to be rid of it inside the command for various reasons: every > command has one of these, and they're expensive in the allocation (at 96 > bytes). There's no reason we have to allocate and free that amount of > space with every command. In theory, the number of these is bounded at >

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 09:09 -0600, James Bottomley wrote: > > What other drivers do is DMA to their own allocation and then memcpy to > > the sense buffer. > > > > There is a movement to allocate the sense data as its own sg list, but > > I don't think that patch has even been posted yet. > > I'

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 05:32 -0700, Matthew Wilcox wrote: > On Mon, Nov 19, 2007 at 04:35:23PM +1100, Benjamin Herrenschmidt wrote: > > The other one I'm hitting now is that the SCSI layer nowadays embeds the > > 'nowadays'? It has always been so. Wasn't it kmalloc'ed at one point ? > > sense_b

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Benjamin Herrenschmidt
On Mon, 2007-11-19 at 00:38 -0800, David Miller wrote: > From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> > Date: Mon, 19 Nov 2007 16:35:23 +1100 > > > I'm not sure what is the best way to fix that. Internally, I've done > > some test whacking some cacheline_aligned in the scsi_cmnd data > >

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread James Bottomley
On Mon, 2007-11-19 at 05:32 -0700, Matthew Wilcox wrote: > On Mon, Nov 19, 2007 at 04:35:23PM +1100, Benjamin Herrenschmidt wrote: > > The other one I'm hitting now is that the SCSI layer nowadays embeds the > > 'nowadays'? It has always been so. > > > sense_buffer inside the scsi_cmnd structur

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread Matthew Wilcox
On Mon, Nov 19, 2007 at 04:35:23PM +1100, Benjamin Herrenschmidt wrote: > The other one I'm hitting now is that the SCSI layer nowadays embeds the 'nowadays'? It has always been so. > sense_buffer inside the scsi_cmnd structure without any kind of > alignment whatsoever. I've been hitting irregu

Re: SCSI breakage on non-cache coherent architectures

2007-11-19 Thread David Miller
From: Benjamin Herrenschmidt <[EMAIL PROTECTED]> Date: Mon, 19 Nov 2007 16:35:23 +1100 > I'm not sure what is the best way to fix that. Internally, I've done > some test whacking some cacheline_aligned in the scsi_cmnd data > structure to verify I no longer get random SLAB corruption when usin

SCSI breakage on non-cache coherent architectures

2007-11-18 Thread Benjamin Herrenschmidt
Hi James ! (Please CC me on replies as I'm not subscribed to linux-scsi) I've been debugging various issues on the PowerPC 44x embedded architecture which happens to have non-coherent PCI DMA. One of the problem I'm hitting is that one really need to enforce kmalloc alignement to cache lines or