RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Pravin Bathija
Benjamin Herrenschmidt wrote: > Do you know many drivers that do config space accesses without using > the config space accessors ? > Such drivers should be banned to oblivion. > Cheers, > Ben. I'm not aware of such drivers in the 2.6.30+ kernel. Thanks, Pravin On Thu, 2009-09-10 at 22:35 -07

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Prodyut Hazarika
> I tried, using our JTAG debugger (BDI3000), to pause operation after > calling dma_alloc_coherent to examine the TLB entry for the memory > returned by the call (which was just past > CONFIG_CONSISTENT_START=0xff10). The TLB list loaded at the time > that I paused operation did not show

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Tom Burns
Hi Ben, Benjamin Herrenschmidt wrote: On Wed, 2009-09-09 at 09:43 -0400, Tom Burns wrote: Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing invalidate_dcache_range() in arch/ppc/kernel/misc.S from __dma_sync(). This OOPses

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Benjamin Herrenschmidt
On Fri, 2009-09-11 at 10:17 +0300, Mikhail Zolotaryov wrote: > Benjamin Herrenschmidt wrote: > > On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: > > > >> Hi Tom, > >> > >> In my case __dma_sync() calls flush_dcache_range() (it's due to > >> alignment) from a tasklet - no OOPS. It u

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-11 Thread Mikhail Zolotaryov
Benjamin Herrenschmidt wrote: On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: Hi Tom, In my case __dma_sync() calls flush_dcache_range() (it's due to alignment) from a tasklet - no OOPS. It uses dcbf instruction instead of dcbi - that's the difference as dcbf is not privileged

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Pravin Bathija
> Stefan Roese wrote: > Correct. IIRC, some PATA driver as Pravin already mentioned. > Cheers, > Stefan Thanks Stefan. The whole intention of the patch/hack (or whatever one might call it :) ) was to avoid rogue drivers from setting pci_cache_line_size to non-zero value even though the underly

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 22:35 -0700, Pravin Bathija wrote: > Thanks Stefan. The whole intention of the patch/hack (or whatever one > might call it :) ) was to avoid rogue drivers from setting > pci_cache_line_size to non-zero value even though the underlying > hardware doesn't support MRM calls. None

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Stefan Roese
On Friday 11 September 2009 07:17:50 Benjamin Herrenschmidt wrote: > On Fri, 2009-09-11 at 07:12 +0200, Stefan Roese wrote: > > It's already there. See commit: > > > > 5ce4b59653b2c2053cd9a011918ac1e4747f24cc > > > > powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata > > Ok, that's another wa

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Fri, 2009-09-11 at 07:12 +0200, Stefan Roese wrote: > > It's already there. See commit: > > 5ce4b59653b2c2053cd9a011918ac1e4747f24cc > > powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata > Ok, that's another way to do it. Will catch nasty drivers who try to write directly rather than c

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Stefan Roese
On Friday 11 September 2009 04:44:34 Benjamin Herrenschmidt wrote: > On Thu, 2009-09-10 at 13:30 -0700, Pravin Bathija wrote: > > There is also a patch that was submitted for 440EPX a couple of years > > back. The 440EPX SOC causes hangs with Memory Read Multiple (MRM) > > commands. Whether MRM is

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 13:30 -0700, Pravin Bathija wrote: > There is also a patch that was submitted for 440EPX a couple of years > back. The 440EPX SOC causes hangs with Memory Read Multiple (MRM) > commands. Whether MRM is used or not depends on the value of > PCI_CACHE_LINE_SIZE register. I see

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Thu, 2009-09-10 at 15:53 -0400, Tom Burns wrote: > Hi, > > Thank you everyone for your help. > > I've been looking into the other dma/pci API calls (dma_alloc_coherent, > pci_alloc_consistent). I don't see how either of these return memory > mapped to a TLB with the I bit set to 1 in kernel

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Wed, 2009-09-09 at 17:40 +0300, Mikhail Zolotaryov wrote: > Hi Tom, > > In my case __dma_sync() calls flush_dcache_range() (it's due to > alignment) from a tasklet - no OOPS. It uses dcbf instruction instead of > dcbi - that's the difference as dcbf is not privileged. What it calls depends o

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Wed, 2009-09-09 at 10:10 -0400, Tom Burns wrote: > Hi Mikhail, > > Sorry, this DMA code is in a tasklet. Are you suggesting the processor > is in supervisor mode at that time? Calling pci_dma_sync_sg_for_cpu() > from the tasklet context is what generates the OOPS. The entire oops is > as

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Benjamin Herrenschmidt
On Wed, 2009-09-09 at 09:43 -0400, Tom Burns wrote: > Hi, > > With the default config for the Sequoia board on 2.6.24, calling > pci_dma_sync_sg_for_cpu() results in executing > invalidate_dcache_range() in arch/ppc/kernel/misc.S from __dma_sync(). > This OOPses on PPC440 since it tries to call

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Pravin Bathija
> Tom Burns wrote > Hi, > > Thank you everyone for your help. > > I've been looking into the other dma/pci API calls (dma_alloc_coherent, > pci_alloc_consistent). I don't see how either of these return memory > mapped to a TLB with the I bit set to 1 in kernel 2.6.24. In our > kernel > code, t

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-10 Thread Tom Burns
Hi, Thank you everyone for your help. I've been looking into the other dma/pci API calls (dma_alloc_coherent, pci_alloc_consistent). I don't see how either of these return memory mapped to a TLB with the I bit set to 1 in kernel 2.6.24. In our kernel code, the only use of the PPC44x_TLB_I d

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Mikhail Zolotaryov
Hi Tom, In my case __dma_sync() calls flush_dcache_range() (it's due to alignment) from a tasklet - no OOPS. It uses dcbf instruction instead of dcbi - that's the difference as dcbf is not privileged. Tom Burns wrote: Hi Mikhail, Sorry, this DMA code is in a tasklet. Are you suggesting the

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Tom Burns
Hi Mikhail, Sorry, this DMA code is in a tasklet. Are you suggesting the processor is in supervisor mode at that time? Calling pci_dma_sync_sg_for_cpu() from the tasklet context is what generates the OOPS. The entire oops is as follows, if it's relevant: Oops: kernel access of bad area, s

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Tom Burns
Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing invalidate_dcache_range() in arch/ppc/kernel/misc.S from __dma_sync(). This OOPses on PPC440 since it tries to call directly the assembly instruction dcbi, which can only be exe

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Mikhail Zolotaryov
Hi Tom, possible solution could be to use tasklet to perform DMA-related job (as in most cases DMA transfer is interrupt driven - makes sense). Tom Burns wrote: Hi, With the default config for the Sequoia board on 2.6.24, calling pci_dma_sync_sg_for_cpu() results in executing invalidate_dc

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-09 Thread Mikhail Zolotaryov
Hi, Why manage cache lines manually, if appropriate code is a part of __dma_sync / dma_sync_single_for_device of DMA API ? (implies CONFIG_NOT_COHERENT_CACHE enabled, as default for Sequoia Board) Prodyut Hazarika wrote: Hi Adam, Yes, I am using the 440EPx (same as the sequoia board).

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 16:00 -0400, Adam Zilkie wrote: > We are using pci_alloc_consistent() Then your flush should have no effect since pci_alloc_consistent will return I=1 mapped memory, unless you don't have CONFIG_NOT_COHERENT_CACHE for some reason. Cheers, Ben. > Adam > > On Tue, 2009-09-08

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 15:30 -0400, Adam Zilkie wrote: > All, > > We have found that using flush_dcache_range() after each DMA solves the > problem. Ideally, we'd like to be able to allocate the virtual page in > cache inhibited memory to avoid the performance loss from all the flush > calls. To do

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Benjamin Herrenschmidt
On Tue, 2009-09-08 at 14:01 -0400, Adam Zilkie wrote: > Hi Prodyut, > > Yes, I am using the 440EPx (same as the sequoia board). > Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using > the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) > end up being partial

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
We are using pci_alloc_consistent() Adam On Tue, 2009-09-08 at 12:56 -0700, Prodyut Hazarika wrote: > > We have found that using flush_dcache_range() after each DMA solves > the > > problem. Ideally, we'd like to be able to allocate the virtual page in > > cache inhibited memory to avoid the perf

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Prodyut Hazarika
> We have found that using flush_dcache_range() after each DMA solves the > problem. Ideally, we'd like to be able to allocate the virtual page in > cache inhibited memory to avoid the performance loss from all the flush > calls. To do this, we'd have to change our TLB sizes and reserve a TLB > in

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
All, We have found that using flush_dcache_range() after each DMA solves the problem. Ideally, we'd like to be able to allocate the virtual page in cache inhibited memory to avoid the performance loss from all the flush calls. To do this, we'd have to change our TLB sizes and reserve a TLB in memo

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Prodyut Hazarika
Hi Adam, > Yes, I am using the 440EPx (same as the sequoia board). > Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using > the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) > end up being partially corrupted when we try to parse the data in the > virtual p

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-08 Thread Adam Zilkie
Hi Prodyut, Yes, I am using the 440EPx (same as the sequoia board). Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing) end up being partially corrupted when we try to parse the data in the virtual page. We

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-06 Thread Benjamin Herrenschmidt
f Of Chris Pringle > Sent: Donnerstag, 3. September 2009 10:05 > To: azil...@datacast.com > Cc: Tom Burns; Andrea Zypchen; linuxppc-dev@lists.ozlabs.org > Subject: Re: AW: PowerPC PCI DMA issues (prefetch/coherency?) > > Hi Adam, > > If you have a look in include/asm-

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Prodyut Hazarika
Hi Adam, > Are you sure there is L2 cache on the 440? It depends on the SoC you are using. SoC like 460EX (Canyonlands board) have L2Cache. It seems you are using a Sequoia board, which has a 440EPx SoC. 440EPx has a 440 cpu core, but no L2Cache. Could you please tell me which SoC you are using?

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Josh Boyer
On Thu, Sep 03, 2009 at 12:04:50PM -0400, Adam Zilkie wrote: >Ben, > >Thanks for your info. > >Are you sure there is L2 cache on the 440? It depends on which 440 SoC you have. It also depends on that being configured in the kernel even if it does exist. >I am seeing this problem with our custom

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Adam Zilkie
Ben, Thanks for your info. Are you sure there is L2 cache on the 440? I am seeing this problem with our custom IDE driver which is based on pretty old code. Our driver uses pci_alloc_consistent() to allocate the physical DMA memory and alloc_pages() to allocate a virtual page. It then uses pci_m

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Adam Zilkie
Chris, I noticed the following comment in pgtable.h: * - CACHE COHERENT bit (M) has no effect on PPC440 core, because it * doesn't support SMP. So we can use this as software bit, like * DIRTY. And _PAGE_COHERENT is not defined for the 44x (giving a compile error when I add it the _PA

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Chris Pringle
3. September 2009 10:05 To: azil...@datacast.com Cc: Tom Burns; Andrea Zypchen; linuxppc-dev@lists.ozlabs.org Subject: Re: AW: PowerPC PCI DMA issues (prefetch/coherency?) Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_P

RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Wrobel Heinz-R39252
AW: PowerPC PCI DMA issues (prefetch/coherency?) Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED) #en

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Benjamin Herrenschmidt
On Thu, 2009-09-03 at 09:05 +0100, Chris Pringle wrote: > Hi Adam, > > If you have a look in include/asm-ppc/pgtable.h for the following section: > #ifdef CONFIG_44x > #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) > #else > #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCE

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-03 Thread Chris Pringle
Hi Adam, If you have a look in include/asm-ppc/pgtable.h for the following section: #ifdef CONFIG_44x #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_GUARDED) #else #define _PAGE_BASE(_PAGE_PRESENT | _PAGE_ACCESSED) #endif Try adding _PAGE_COHERENT to the appropriate line above

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-09-02 Thread Adam Zilkie
Hi Chris, I am having a problem similar to what you described in this discussion. We are using the ppc arch with 2.6.24 with CONFIG_SEQUOIA with compiles arch/ppc/kernel/head_44x.c (quite different from /arch/powerpc/kernel/head_32.S). I would like to apply your backporting patch to this architect

Re: AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-06-29 Thread Chris Pringle
Hi Sergej, I've attached the patch used to fix this issue. Both the patch to pgtable32.h and head_32.S are required in order to make it work. The change to pgtable32.h ensures that all pages are marked cache coherent (results in setting the M bit). The change to head_32.S ensures that the M b

AW: PowerPC PCI DMA issues (prefetch/coherency?)

2009-06-22 Thread Sergej.Stepanov
>The other part of the fix is in asm-powerpc/pgtable32.h. _PAGE_BASE >needs _PAGE_COHERENT in order to work correctly, and in fact there is >now a comment in there to that affect in 2.6.29. Backporting that change >has made it work on 2.6.26. Both this patch, and the fix to head_32.S >are needed fo