On Tue, 2009-09-08 at 16:00 -0400, Adam Zilkie wrote: > We are using pci_alloc_consistent()
Then your flush should have no effect since pci_alloc_consistent will return I=1 mapped memory, unless you don't have CONFIG_NOT_COHERENT_CACHE for some reason. Cheers, Ben. > Adam > > On Tue, 2009-09-08 at 12:56 -0700, Prodyut Hazarika wrote: > > > We have found that using flush_dcache_range() after each DMA solves > > the > > > problem. Ideally, we'd like to be able to allocate the virtual page in > > > cache inhibited memory to avoid the performance loss from all the > > flush > > > calls. To do this, we'd have to change our TLB sizes and reserve a TLB > > > in memory as cache inhibited (using the 'I' bit). Will update if this > > > works as well. Thanks for your help in this. > > > > Aren't you using dma_alloc_coherent to get buffers that are shared > > between CPU and external devices? > > > > Thanks > > Prodyut > > > > On Tue, 2009-09-08 at 11:59 -0700, Prodyut Hazarika wrote: > > > Hi Adam, > > > > > > > Yes, I am using the 440EPx (same as the sequoia board). > > > > Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus > > > (using > > > > the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on > > timing) > > > > end up being partially corrupted when we try to parse the data in > > the > > > > virtual page. We have confirmed the data is good before the PCI-IDE > > > > bridge. We are creating two 8K pages and map them to physical DMA > > > memory > > > > using single-entry scatter/gather structs. When a DMA block is > > > > corrupted, we see a random portion of it (always a multiple of > > 16byte > > > > cache lines) is overwritten with old data from the last time the > > > buffer > > > > was used. > > > > > > This looks like a cache coherency problem. > > > Can you ensure that the TLB entries corresponding to the DMA region > > has > > > the CacheInhibit bit set. > > > You will need a BDI connected to your system. > > > > > > Also, you will need to invalidate and flush the lines appropriately, > > > since in 440 cores, > > > L1Cache coherency is managed entirely by software. > > > Please look at drivers/net/ibm_newemac/mal.c and core.c for example on > > > how to do it. > > > > > > Thanks > > > Prodyut > > > > > > On Thu, 2009-09-03 at 13:27 -0700, Prodyut Hazarika wrote: > > > > Hi Adam, > > > > > > > > > Are you sure there is L2 cache on the 440? > > > > > > > > It depends on the SoC you are using. SoC like 460EX (Canyonlands > > > board) > > > > have L2Cache. > > > > It seems you are using a Sequoia board, which has a 440EPx SoC. > > 440EPx > > > > has a 440 cpu core, but no L2Cache. > > > > Could you please tell me which SoC you are using? > > > > You can also refer to the appropriate dts file to see if there is > > L2C. > > > > For example, in canyonlands.dts (460EX based board), we have the L2C > > > > entry. > > > > L2C0: l2c { > > > > ... > > > > } > > > > > > > > >I am seeing this problem with our custom IDE driver which is based > > on > > > > > > > >pretty old code. Our driver uses pci_alloc_consistent() to allocate > > > the > > > > > > > > >physical DMA memory and alloc_pages() to allocate a virtual page. > > It > > > > >then uses pci_map_sg() to map to a scatter/gather buffer. Perhaps I > > > > > > >should convert these to the DMA API calls as you suggest. > > > > > > > > Could you give more details on the consistency problem? It is a good > > > > idea to change to the new DMA APIs, but pci_alloc_consistent() > > should > > > > work too > > > > > > > > Thanks > > > > Prodyut > > > > > > > > On Thu, 2009-09-03 at 19:57 +1000, Benjamin Herrenschmidt wrote: > > > > > On Thu, 2009-09-03 at 09:05 +0100, Chris Pringle wrote: > > > > > > Hi Adam, > > > > > > > > > > > > If you have a look in include/asm-ppc/pgtable.h for the > > following > > > > section: > > > > > > #ifdef CONFIG_44x > > > > > > #define _PAGE_BASE (_PAGE_PRESENT | _PAGE_ACCESSED | > > > > _PAGE_GUARDED) > > > > > > #else > > > > > > #define _PAGE_BASE (_PAGE_PRESENT | _PAGE_ACCESSED) > > > > > > #endif > > > > > > > > > > > > Try adding _PAGE_COHERENT to the appropriate line above and see > > if > > > > that > > > > > > fixes your issue - this causes the 'M' bit to be set on the page > > > > which > > > > > > sure enforce cache coherency. If it doesn't, you'll need to > > check > > > > the > > > > > > 'M' bit isn't being masked out in head_44x.S (it was originally > > > > masked > > > > > > out on arch/powerpc, but was fixed in later kernels when the > > cache > > > > > > > > > coherency issues with non-SMP systems were resolved). > > > > > > > > > > I have some doubts about the usefulness of doing that for 4xx. > > > AFAIK, > > > > > the 440 core just ignores M. > > > > > > > > > > The problem lies probably elsewhere. Maybe the L2 cache coherency > > > > isn't > > > > > enabled or not working ? > > > > > > > > > > The L1 cache on 440 is simply not coherent, so drivers have to > > make > > > > sure > > > > > they use the appropriate DMA APIs which will do cache flushing > > when > > > > > needed. > > > > > > > > > > Adam, what driver is causing you that sort of problems ? > > > > > > > > > > Cheers, > > > > > Ben. > > > > > > > > > > > > > -- > > > Adam Zilkie > > > Software Designer, > > > International Datacasting Corp. > > > > > > This message and the documents attached hereto are intended only for > > the > > > addressee and may contain privileged or confidential information. Any > > > unauthorized disclosure is strictly prohibited. If you have received > > > this message in error, please notify us immediately so that we may > > > correct our internal records. Please then delete the original message. > > > Thank you. > > > -------------------------------------------------------- > > > > > > CONFIDENTIALITY NOTICE: This e-mail message, including any > > attachments, is for the sole use of the intended recipient(s) and > > contains information that is confidential and proprietary to > > AppliedMicro Corporation or its subsidiaries. It is to be used solely > > for the purpose of furthering the parties' business relationship. All > > unauthorized review, use, disclosure or distribution is prohibited. If > > you are not the intended recipient, please contact the sender by reply > > e-mail and destroy all copies of the original message. > > > _______________________________________________ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev