RE: 440SPe/Katmai PCIe problems
-Original Message- From: Benjamin Herrenschmidt [mailto:b...@kernel.crashing.org] Sent: Tuesday, May 11, 2010 8:03 PM To: Stefan Roese Cc: linuxppc-dev; Pravin Bathija Subject: Re: 440SPe/Katmai PCIe problems On Wed, 2010-05-12 at 12:40 +1000, Benjamin Herrenschmidt wrote: Hi Stefan ! (Or somebody from AMCC) I noticed we still have this old problem on PCIe where some cards don't seem to work. The link trains but they don't respond on config space. This is the case for example of that Sunix quad USB card. It seems to be related to the kind of TI PCIe - PCI bridge on these though. Have anybody tracked down the root cause of the problem ? Some say that it could be that the bridge doesn't cope with the double reset (uboot then kernel). Would it be possible to tell uboot to avoid touching the PCIe interface completely so the kernel is the first one to get in ? Actually... the card works in the Canyonlands using the latest u-boot from ftp.denx.de (u-boot-nand.bin-2009.11.1), though interestingly enough, this u-boot fails to detect an XGI Z11 video card that I have, whichever slot I put it in. in fact, the kernel itself detects the card and times out trying to get a link. It works with whatever u-boot I used to have in there (and no longer do ... oops :-) Cheers, Ben. Ben, Thanks for bringing this to our attention. We'll research this issue. Regards, Pravin ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64 bit resources.
Hi Wolfgang,
-Original Message-
From: Wolfgang Denk [mailto:w...@denx.de]
Sent: Thursday, December 03, 2009 12:56 AM
To: Pravin Bathija; Benjamin Herrenschmidt; Desai, Kashyap
Cc: linux-s...@vger.kernel.org; linuxppc-...@ozlabs.org;
eric.mo...@lsi.com
Subject: Re: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64
bit resources.
Dear Pravin Bathija,
In message 1259805106-23636-1-git-send-email-pbath...@amcc.com you
wrote:
Powerpc 44x uses 36 bit real address while the real address
defined
in MPT Fusion driver is of type 32 bit. This causes ioremap to
fail and driver
fails to initialize. This fix changes the data types
representing
the real
address from unsigned long 32-bit types to resource_size_t which
is 64-bit. The
driver has been tested, the disks get discovered correctly and
can do IO.
...
--- a/drivers/message/fusion/mptbase.c
+++ b/drivers/message/fusion/mptbase.c
@@ -1511,7 +1511,7 @@ mpt_mapresources(MPT_ADAPTER *ioc)
{
u8 __iomem *mem;
int ii;
- unsigned longmem_phys;
+ resource_size_t mem_phys;
unsigned longport;
u32 msize;
u32 psize;
I'm not sure if this one-liner really covers all the related issues.
We submitted a similar (but apparently more complete) patch more than
a year ago. Dunno why it has never been picked up. See
http://thread.gmane.org/gmane.linux.scsi/46082 for reference.
I submitted a patch on similar lines several weeks ago and it wasn't
accepted on grounds that it was too tied to the powerpc platform. Below
is a link
http://article.gmane.org/gmane.linux.scsi/55794
From: Yuri Tikhonov y...@emcraft.com
To: linux-s...@vger.kernel.org
Subject: [PATCH] mptbase: use resource_size_t instead of unsigned long
and u32
Date: Thu, 13 Nov 2008 11:33:16 +0300
Hello,
The following patch adds using resource_size_t for the
pci_resource_start()/pci_resource_len() return values. This
makes mptbase driver work correctly on 32 bit systems with
64 bit resources (e.g. PPC440SPe).
Do some minor cleanups in mpt_mapresources() as well.
Signed-off-by: Yuri Tikhonov y...@emcraft.com
Signed-off-by: Ilya Yanok ya...@emcraft.com
---
drivers/message/fusion/mptbase.c | 38
++
drivers/message/fusion/mptbase.h |5 +++--
2 files changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/message/fusion/mptbase.c
b/drivers/message/fusion/mptbase.c
index d6a0074..9daf844 100644
--- a/drivers/message/fusion/mptbase.c
+++ b/drivers/message/fusion/mptbase.c
@@ -1488,11 +1488,12 @@ static int
mpt_mapresources(MPT_ADAPTER *ioc)
{
u8 __iomem *mem;
+ u8 __iomem *port;
int ii;
- unsigned longmem_phys;
- unsigned longport;
- u32 msize;
- u32 psize;
+ resource_size_t mem_phys;
+ resource_size_t port_phys;
+ resource_size_t msize;
+ resource_size_t psize;
u8 revision;
int r = -ENODEV;
struct pci_dev *pdev;
@@ -1530,13 +1531,13 @@ mpt_mapresources(MPT_ADAPTER *ioc)
}
mem_phys = msize = 0;
- port = psize = 0;
+ port_phys = psize = 0;
for (ii = 0; ii DEVICE_COUNT_RESOURCE; ii++) {
if (pci_resource_flags(pdev, ii)
PCI_BASE_ADDRESS_SPACE_IO) {
if (psize)
continue;
/* Get I/O space! */
- port = pci_resource_start(pdev, ii);
+ port_phys = pci_resource_start(pdev, ii);
psize = pci_resource_len(pdev, ii);
} else {
if (msize)
@@ -1546,11 +1547,8 @@ mpt_mapresources(MPT_ADAPTER *ioc)
msize = pci_resource_len(pdev, ii);
}
}
- ioc-mem_size = msize;
- mem = NULL;
/* Get logical ptr for PciMem0 space */
- /*mem = ioremap(mem_phys, msize);*/
mem = ioremap(mem_phys, msize);
if (mem == NULL) {
printk(MYIOC_s_ERR_FMT : ERROR - Unable to map adapter
@@ -1558,14 +1556,24 @@ mpt_mapresources(MPT_ADAPTER *ioc)
return -EINVAL;
}
ioc-memmap = mem;
- dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem = %p, mem_phys =
%lx\n,
- ioc-name, mem, mem_phys));
+ dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem=%p,
mem_phys=%llx\n,
+ ioc-name, mem, (u64)mem_phys));
ioc-mem_phys = mem_phys;
ioc-chip = (SYSIF_REGS __iomem *)mem;
/* Save Port IO values in case we need to do downloadboot */
- ioc-pio_mem_phys = port;
+ port = ioremap(port_phys, psize);
+ if (port == NULL) {
+ printk(MYIOC_s_ERR_FMT : ERROR - Unable to map adapter
+ port!\n, ioc-name
[PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64 bit resources.
Powerpc 44x uses 36 bit real address while the real address defined
in MPT Fusion driver is of type 32 bit. This causes ioremap to fail and
driver
fails to initialize. This fix changes the data types representing the real
address from unsigned long 32-bit types to resource_size_t which is 64-bit.
The
driver has been tested, the disks get discovered correctly and can do IO.
Signed-off-by: Pravin Bathija pbath...@amcc.com
Acked-by: Feng Kan f...@amcc.com
Acked-by: Fushen Chen fc...@amcc.com
Acked-by: Loc Ho l...@amcc.com
Acked-by: Tirumala Reddy Marri tma...@amcc.com
Acked-by: Victor Gallardo vgalla...@amcc.com
---
drivers/message/fusion/mptbase.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/message/fusion/mptbase.c b/drivers/message/fusion/mptbase.c
index 5d496a9..9f14a60 100644
--- a/drivers/message/fusion/mptbase.c
+++ b/drivers/message/fusion/mptbase.c
@@ -1511,7 +1511,7 @@ mpt_mapresources(MPT_ADAPTER *ioc)
{
u8 __iomem *mem;
int ii;
- unsigned longmem_phys;
+ resource_size_t mem_phys;
unsigned longport;
u32 msize;
u32 psize;
--
1.5.5
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
[PATCH] [PPC4xx] Fix device tree dts file for katmai board.
Description: Set PCI-E node inbound DMA ranges size to 4GB for correct boot up of katmai. Including only changes for PCI-E DMA ranges as suggested by Stefan. Signed-off-by: Pravin Bathija pbath...@amcc.com Acked-by: Feng Kan f...@amcc.com Acked-by: Prodyut Hazarika phazar...@amcc.com Acked-by: Loc Ho l...@amcc.com Acked-by: Tirumala Reddy Marri tma...@amcc.com Acked-by: Victor Gallardo vgalla...@amcc.com --- arch/powerpc/boot/dts/katmai.dts | 22 +++--- 1 files changed, 11 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/boot/dts/katmai.dts b/arch/powerpc/boot/dts/katmai.dts index 077819b..d2595b2 100644 --- a/arch/powerpc/boot/dts/katmai.dts +++ b/arch/powerpc/boot/dts/katmai.dts @@ -245,8 +245,8 @@ ranges = 0x0200 0x 0x8000 0x000d 0x8000 0x 0x8000 0x0100 0x 0x 0x000c 0x0800 0x 0x0001; - /* Inbound 2GB range starting at 0 */ - dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x0 0x8000; + /* Inbound 4GB range starting at 0 */ + dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x1 0x; /* This drives busses 0 to 0xf */ bus-range = 0x0 0xf; @@ -289,10 +289,10 @@ ranges = 0x0200 0x 0x8000 0x000e 0x 0x 0x8000 0x0100 0x 0x 0x000f 0x8000 0x 0x0001; - /* Inbound 2GB range starting at 0 */ - dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x0 0x8000; + /* Inbound 4GB range starting at 0 */ + dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x1 0x; - /* This drives busses 10 to 0x1f */ + /* This drives busses 0x10 to 0x1f */ bus-range = 0x10 0x1f; /* Legacy interrupts (note the weird polarity, the bridge seems @@ -330,10 +330,10 @@ ranges = 0x0200 0x 0x8000 0x000e 0x8000 0x 0x8000 0x0100 0x 0x 0x000f 0x8001 0x 0x0001; - /* Inbound 2GB range starting at 0 */ - dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x0 0x8000; + /* Inbound 4GB range starting at 0 */ + dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x1 0x; - /* This drives busses 10 to 0x1f */ + /* This drives busses 0x20 to 0x2f */ bus-range = 0x20 0x2f; /* Legacy interrupts (note the weird polarity, the bridge seems @@ -371,10 +371,10 @@ ranges = 0x0200 0x 0x8000 0x000f 0x 0x 0x8000 0x0100 0x 0x 0x000f 0x8002 0x 0x0001; - /* Inbound 2GB range starting at 0 */ - dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x0 0x8000; + /* Inbound 4GB range starting at 0 */ + dma-ranges = 0x4200 0x0 0x0 0x0 0x0 0x1 0x; - /* This drives busses 10 to 0x1f */ + /* This drives busses 0x30 to 0x3f */ bus-range = 0x30 0x3f; /* Legacy interrupts (note the weird polarity, the bridge seems -- 1.5.5 ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64 bit resources.
-Original Message-
From: Benjamin Herrenschmidt [mailto:b...@kernel.crashing.org]
Sent: Thursday, November 05, 2009 12:00 PM
To: James Bottomley
Cc: Josh Boyer; eric.mo...@lsi.com; Pravin Bathija; linux-
s...@vger.kernel.org; linuxppc-...@ozlabs.org
Subject: Re: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64
bit resources.
On Thu, 2009-11-05 at 10:07 -0600, James Bottomley wrote:
ioc-memmap = mem;
-dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem = %p,
mem_phys = %lx\n,
-ioc-name, mem, mem_phys));
+dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem = %p,
mem_phys = %llx\n,
+ioc-name, mem, (u64)mem_phys));
ioc-mem_phys = mem_phys;
ioc-chip = (SYSIF_REGS __iomem *)mem;
/* Save Port IO values in case we need to do downloadboot
*/
-ioc-pio_mem_phys = port;
+port = ioremap(port_phys, psize);
+if (port == NULL) {
+printk(MYIOC_s_ERR_FMT : ERROR - Unable to map
adapter
+ port !\n, ioc-name);
+return -EINVAL;
So this looks problematic on a few platforms ... what happens to
platforms that have no IO space? They automatically fail here and it
looks like the adapter never attaches.
Yup, that part of the patch looks wrong.
However, a mechanical replacement of unsigned long's with
resource_size_t to hold physical addresses should be fine despite the
lack of feedback from LSI.
Pravin, that ioremap definitely seems like it has nothing to do there,
port IO is already remapped for you by the core PCI code and should
work
as is. Please respin without that change.
Cheers,
Ben.
Thanks for the input. Will make the suggested changes and re-submit patch.
Regards,
Pravin
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64 bit resources.
-Original Message-
From: Benjamin Herrenschmidt [mailto:b...@kernel.crashing.org]
Sent: Thu 11/5/2009 12:00 PM
To: James Bottomley
Cc: Josh Boyer; eric.mo...@lsi.com; Pravin Bathija; linux-s...@vger.kernel.org;
linuxppc-...@ozlabs.org
Subject: Re: [PATCH] [SCSI] mpt fusion: Fix 32 bit platforms with 64 bit
resources.
On Thu, 2009-11-05 at 10:07 -0600, James Bottomley wrote:
ioc-memmap = mem;
- dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem = %p, mem_phys = %lx\n,
- ioc-name, mem, mem_phys));
+ dinitprintk(ioc, printk(MYIOC_s_INFO_FMT mem = %p, mem_phys = %llx\n,
+ ioc-name, mem, (u64)mem_phys));
ioc-mem_phys = mem_phys;
ioc-chip = (SYSIF_REGS __iomem *)mem;
/* Save Port IO values in case we need to do downloadboot */
- ioc-pio_mem_phys = port;
+ port = ioremap(port_phys, psize);
+ if (port == NULL) {
+ printk(MYIOC_s_ERR_FMT : ERROR - Unable to map adapter
+ port !\n, ioc-name);
+ return -EINVAL;
So this looks problematic on a few platforms ... what happens to
platforms that have no IO space? They automatically fail here and it
looks like the adapter never attaches.
Yup, that part of the patch looks wrong.
However, a mechanical replacement of unsigned long's with
resource_size_t to hold physical addresses should be fine despite the
lack of feedback from LSI.
Pravin, that ioremap definitely seems like it has nothing to do there,
port IO is already remapped for you by the core PCI code and should work
as is. Please respin without that change.
Cheers,
Ben.
Thanks for the input. Will make the suggested changes and re-submit the patch.
Regards,
Pravin
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)
Stefan Roese wrote: Correct. IIRC, some PATA driver as Pravin already mentioned. Cheers, Stefan Thanks Stefan. The whole intention of the patch/hack (or whatever one might call it :) ) was to avoid rogue drivers from setting pci_cache_line_size to non-zero value even though the underlying hardware doesn't support MRM calls. Nonetheless this approach works only if the drivers use the kernel API for PCI config space access provided by the powerpc platform driver. Regards, Pravin On Friday 11 September 2009 07:17:50 Benjamin Herrenschmidt wrote: On Fri, 2009-09-11 at 07:12 +0200, Stefan Roese wrote: It's already there. See commit: 5ce4b59653b2c2053cd9a011918ac1e4747f24cc powerpc/4xx: Workaround for PPC440EPx/GRx PCI_28 Errata Ok, that's another way to do it. Will catch nasty drivers who try to write directly rather than clear pci_cache_line_size I suppose... ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)
Benjamin Herrenschmidt wrote: Do you know many drivers that do config space accesses without using the config space accessors ? Such drivers should be banned to oblivion. Cheers, Ben. I'm not aware of such drivers in the 2.6.30+ kernel. Thanks, Pravin On Thu, 2009-09-10 at 22:35 -0700, Pravin Bathija wrote: Thanks Stefan. The whole intention of the patch/hack (or whatever one might call it :) ) was to avoid rogue drivers from setting pci_cache_line_size to non-zero value even though the underlying hardware doesn't support MRM calls. Nonetheless this approach works only if the drivers use the kernel API for PCI config space access provided by the powerpc platform driver. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
RE: AW: PowerPC PCI DMA issues (prefetch/coherency?)
Tom Burns wrote
Hi,
Thank you everyone for your help.
I've been looking into the other dma/pci API calls
(dma_alloc_coherent,
pci_alloc_consistent). I don't see how either of these return memory
mapped to a TLB with the I bit set to 1 in kernel 2.6.24. In our
kernel
code, the only use of the PPC44x_TLB_I define is in head_44x.S in
_start. We have CONFIG_NON_COHERENT_CACHE enabled.
We changed our code to use dma_alloc_coherent, removed our manual
cacheline flushing, and saw the corrupted data return. To me this
means
dma_alloc_coherent cannot be setting the I=1 bit in the TLB entry.
I tried, using our JTAG debugger (BDI3000), to pause operation after
calling dma_alloc_coherent to examine the TLB entry for the memory
returned by the call (which was just past
CONFIG_CONSISTENT_START=0xff10). The TLB list loaded at the time
that I paused operation did not show a mapping for this area. I guess
the kernel swaps TLB entries on the fly so it isn't limited to only 64
entries? I will try to sleep in the same context as the
dma_alloc_coherent call to try to catch the TLB entry while loaded to
see if it has the I bit set.
If that fails, any ideas?
Thanks,
Tom Burns
International Datacasting Corporation
There is also a patch that was submitted for 440EPX a couple of years
back. The 440EPX SOC causes hangs with Memory Read Multiple (MRM)
commands. Whether MRM is used or not depends on the value of
PCI_CACHE_LINE_SIZE register. I see that the changes are no longer
present in linux 2.6.30+ kernels. Although the patch certainly resolved
the hang issue with Silicon Image 680 PATA card as the 680 driver
attempts to use MRM commands - I don't know if it would resolve the data
corruption issue. It is certainly worth trying in my opinion. Below is a
link to the patch submission:
http://git.denx.de/?p=linux-2.6-denx.git;a=commit;h=cffefde924123e685327
48dd58fcb780eab5e219
Mikhail Zolotaryov wrote:
Hi Tom,
possible solution could be to use tasklet to perform DMA-related job
(as in most cases DMA transfer is interrupt driven - makes sense).
Tom Burns wrote:
Hi,
With the default config for the Sequoia board on 2.6.24, calling
pci_dma_sync_sg_for_cpu() results in executing
invalidate_dcache_range() in arch/ppc/kernel/misc.S from
__dma_sync(). This OOPses on PPC440 since it tries to call
directly
the assembly instruction dcbi, which can only be executed in
supervisor mode. We tried that before resorting to manual cache
line
management with usermode-safe assembly calls.
Regards,
Tom Burns
International Datacasting Corporation
Mikhail Zolotaryov wrote:
Hi,
Why manage cache lines manually, if appropriate code is a part of
__dma_sync / dma_sync_single_for_device of DMA API ? (implies
CONFIG_NOT_COHERENT_CACHE enabled, as default for Sequoia Board)
Prodyut Hazarika wrote:
Hi Adam,
Yes, I am using the 440EPx (same as the sequoia board). Our
ideDriver is DMA'ing blocks of 192-byte data over the PCI bus
(using
the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on
timing)
end up being partially corrupted when we try to parse the data
in
the
virtual page. We have confirmed the data is good before the PCI-
IDE
bridge. We are creating two 8K pages and map them to physical
DMA
memory
using single-entry scatter/gather structs. When a DMA block is
corrupted, we see a random portion of it (always a multiple of
16byte
cache lines) is overwritten with old data from the last time the
buffer
was used.
This looks like a cache coherency problem.
Can you ensure that the TLB entries corresponding to the DMA
region
has
the CacheInhibit bit set.
You will need a BDI connected to your system.
Also, you will need to invalidate and flush the lines
appropriately,
since in 440 cores,
L1Cache coherency is managed entirely by software.
Please look at drivers/net/ibm_newemac/mal.c and core.c for
example on
how to do it.
Thanks
Prodyut
On Thu, 2009-09-03 at 13:27 -0700, Prodyut Hazarika wrote:
Hi Adam,
Are you sure there is L2 cache on the 440?
It depends on the SoC you are using. SoC like 460EX (Canyonlands
board)
have L2Cache.
It seems you are using a Sequoia board, which has a 440EPx SoC.
440EPx
has a 440 cpu core, but no L2Cache.
Could you please tell me which SoC you are using?
You can also refer to the appropriate dts file to see if there
is
L2C.
For example, in canyonlands.dts (460EX based board), we have the
L2C
entry.
L2C0: l2c {
...
}
I am seeing this problem with our custom IDE driver which is
based on
pretty old code. Our driver uses pci_alloc_consistent() to
allocate
the
physical DMA memory and alloc_pages() to allocate a virtual
page.
It then uses pci_map_sg() to map to a scatter/gather buffer.
Perhaps I should convert these to the DMA API
