44x: How to increase PCIe space (is it kernel/dts/...)
I've got a design that is 405EXr based so its got 32 bit
address space. We have an FPGA that sits on our pcie bus
and we want it to have 2 BARs with 1G of space and maybe
256MB of space. I cannot figure out what I need to change
in the kernel/dts to make this work.
Right now, if I go over 512MB in the DTS "ranges" parameter
then I'll get an error during kernel boot.
My thought is that the magic happens in the function:
ppc4xx_pciex_port_init_mapping(...)
In there I see it setting up the REGBAH and REGBAL and then
this curious statement:
/* XXX FIXME: Use size from reg property. For now, map * 512M */
dcr_write(port->dcrs, DCRO_PEGPL_CFGMSK, 0xe001);
I figure I need to change that or fix it the right way
(though I don't know how to do that). The other thing is my
dts file. It is based on the amcc kilauea and here is the
part that I am unsure of. The PCIe entry falls under the
PLB which could also be a problem but again, I am not sure.
So basically, is there an easy way to change what I've got
to map 1.25 or 2GB of space?
Thanks
Ayman
PCIE0: pciex@0a000 {
device_type = "pci";
#interrupt-cells = <1>;
#size-cells = <2>;
#address-cells = <3>;
compatible = "ibm,plb-pciex-405ex", "ibm,plb-pciex";
primary;
port = <0x0>; /* port number */
reg = <0xa000 0x2000/* Config space access */
0xef00 0x1000>; /* Registers */
dcr-reg = <0x040 0x020>;
sdr-base = <0x400>;
/* Outbound ranges, one memory and * one IO,
* later cannot be changed
*/
ranges = <0x0200 0x 0x8000 0x9000
0x 0x0800
0x0100 0x 0x 0xe000
0x 0x0001>;
/* Inbound 2GB range starting at 0 * */
dma-ranges = <0x4200 0x0 0x0 0x0 0x0 0x8000>;
/* This drives busses 0x00 to 0x3f * */
bus-range = <0x0 0x3f>;
/* Legacy interrupts (note the weird * polarity, the bridge
seems
* to invert PCIe legacy
* interrupts).
* We are de-swizzling here because
* the numbers are actually for
* port of the root complex virtual
* P2P bridge. But I want
* to avoid putting a node for it in
* the tree, so the numbers
* below are basically de-swizzled
* numbers.
* The real slot is on idsel 0, so
* the swizzling is 1:1
*/
interrupt-map-mask = <0x0 0x0 0x0 0x7>;
interrupt-map = < 0x0 0x0 0x0 0x1 &UIC2 0x0 0x4 /* swizzled int
A */
0x0 0x0 0x0 0x2 &UIC2 0x1 0x4
/* swizzled int B */
0x0 0x0 0x0 0x3 &UIC2 0x2 0x4
/* swizzled int C */
0x0 0x0 0x0 0x4 &UIC2 0x3 0x4
/* swizzled int D */>;
};
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: How to handle cache when I allocate phys memory?
I never did get this to work, and now I am back to it again.
On Fri, Oct 14, 2011 at 09:39:51AM +0200, Benjamin Herrenschmidt wrote:
> On Wed, 2011-10-12 at 16:08 -0500, Ayman El-Khashab wrote:
> > I'm using the 460sx (440 core) so no snooping here. What
> > I've done is reserved the top of memory for my driver. My
> > driver can read/write the memory and I can mmap it just
> > fine. The problem is I want to enable caching on the mmap
> > for performance but I don't know / can't figure out how to
> > tell the kernel to sync the cache after it gets dma data
> > from the device or after i put data into it from user space.
> > I know how to do it from regular devices, but not when I've
> > allocated the physical memory myself. I suppose what I am
> > looking for is something akin to dma_sync_single cpu/device.
> >
> > In my device driver, I am allocating the memory like this,
> > in this case the buffer is about 512MB.
> >
> > vma->vm_flags |= VM_LOCKED | VM_RESERVED;
> >
> > /* map the physical area into one buffer */
> > rc = remap_pfn_range(vma, vma->vm_start,
> > (PHYS_MEM_ADDR)>>PAGE_SHIFT,
> > len, vma->vm_page_prot);
> >
> > Is this going to give me the best performance, or is there
> > something more I can do?
> >
> > Failing that, what is the best way to do this (i need a very
> > large contiguous buffer). it runs in batch mode, so it
> > DMAs, stops, cpu reads, cpu writes, repeat ...
>
> Did you try looking at what the dma_* functions do under the hood and
> call it directly (or reproducing it) ?
>
> Basically it boils down to using dcbf instructions to flush dirty data
> or dcbi to invalidate cache lines.
>
I've reserved (using mem=) memory at the top of my system.
In my case, its the upper 1GB of 2GB total. I've got a
small driver that I've written that maps it into user space
using mmap .. that all works fine. I've also got it caching
and that also works. The problem is that depending on how I
do things, I can get some cache-coherency issues. I know in
the user code where to poke things, but I've tried
everything I can think of and *cannot* get flush_dcache_range
to work for me.
My mapping code in the driver is:
static int my_mmap(struct file *fip, struct
vm_area_struct *vma)
{
int rc;
unsigned long len = vma->vm_end - vma->vm_start;
printk(KERN_DEBUG "mapping %ld bytes\n", len);
vma->vm_page_prot = pgprot_cached(vma->vm_page_prot);
kernel_vp = ioremap(PHYS_MEM_ADDR, 1<<20);
rc = remap_pfn_range(vma, vma->vm_start,
(PHYS_MEM_ADDR)>>PAGE_SHIFT,
len, vma->vm_page_prot);
return (rc < 0 ? rc : 0);
}
I've stripped out some comments but otherwise, this is it.
I've tried both with and without ioremap, both fail in the
same way. I've changed the vma->vm_page_prot a number of
ways. In this example, I had knocked the size of ioremap
(and the flush) to 1MB to see if it was a size issue, but
the kernel gives an error as soon as the first dcbf
instruction is executed in the flush loop.
Then I've got an ioctl to flush
case CACHE_FLUSH:
{
u_int32_t phys_start = phys_mem_addr<<20;
u_int32_t phys_stop = (phys_mem_addr +
phys_mem_size)<<20;
//flush_dcache_range(phys_start, phys_stop-1);
flush_dcache_range(kernel_vp, kernel_vp +
(1<<20));
}
break;
kernel_vp is a virtual pointer from ioremap (only 1MB in
size). The phys_stop and phys_start is the physical address
range, which I think might be wrong. It does not work in
either case anyway.
I'd really like to map 1G, make it cachable and do the flush
and invalidate on demand ... what am I missing? Here is the
kernel dump
##Unable to handle kernel paging request for data at
address 0x4000
##Faulting instruction address: 0xc000c398
##Oops: Kernel access of bad area, sig: 11 [#1]
PowerPC 44x Platform
last sysfs file:
/sys/devices/plb.0/opb.3/4ef600400.i2c/i2c-0/0-0022/gpio/gpio223/value
Modules linked in: tan_mpt2sas tanomem [last unloaded:
tanomem]
NIP: c000c398 LR: f4fd91d4 CTR: 0200
REGS: ebad7dc0 TRAP: 0300 Not tainted
(2.6.37.6-tanisys-sx2-24099)
MSR: 00029000 CR: 48040242 XER: 0001
DEAR: 4000, ESR: 0080
TASK = ebceb0c0[2720] 'testapplication' THREAD: ebad6000
GPR00: 0400 ebad7e70 ebceb0c0 4000 0200 001f
28040244 101132b0
GPR08: 0002d000 f4fd9590 ebbda180 c000c37c 101a47c8
0003 bfa4d229
GPR16: bfd98f84 1003a5f8 101416c0 101414b0 bfa4baa8 bfa4bb1c
bfa4bd00
GPR24: 101a1514 bfa4d69c ebbda180 fff7 c0045402
28040244 28040244
NI
How to handle cache when I allocate phys memory?
I'm using the 460sx (440 core) so no snooping here. What I've done is reserved the top of memory for my driver. My driver can read/write the memory and I can mmap it just fine. The problem is I want to enable caching on the mmap for performance but I don't know / can't figure out how to tell the kernel to sync the cache after it gets dma data from the device or after i put data into it from user space. I know how to do it from regular devices, but not when I've allocated the physical memory myself. I suppose what I am looking for is something akin to dma_sync_single cpu/device. In my device driver, I am allocating the memory like this, in this case the buffer is about 512MB. vma->vm_flags |= VM_LOCKED | VM_RESERVED; /* map the physical area into one buffer */ rc = remap_pfn_range(vma, vma->vm_start, (PHYS_MEM_ADDR)>>PAGE_SHIFT, len, vma->vm_page_prot); Is this going to give me the best performance, or is there something more I can do? Failing that, what is the best way to do this (i need a very large contiguous buffer). it runs in batch mode, so it DMAs, stops, cpu reads, cpu writes, repeat ... thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [v2 PATCH 1/1] powerpc/4xx: enable and fix pcie gen1/gen2 on the 460sx
On Mon, Jul 18, 2011 at 02:01:15PM +1000, Tony Breeds wrote: > On Fri, Jul 15, 2011 at 11:40:27AM -0500, Ayman Elkhashab wrote: > > > @@ -1582,8 +1628,8 @@ static int __init ppc4xx_setup_one_pciex_POM(struct > > ppc4xx_pciex_port *port, > > dcr_write(port->dcrs, DCRO_PEGPL_OMR2BAH, lah); > > dcr_write(port->dcrs, DCRO_PEGPL_OMR2BAL, lal); > > dcr_write(port->dcrs, DCRO_PEGPL_OMR2MSKH, 0x7fff); > > - /* Note that 3 here means enabled | single region */ > > - dcr_write(port->dcrs, DCRO_PEGPL_OMR2MSKL, sa | 3); > > + dcr_write(port->dcrs, DCRO_PEGPL_OMR2MSKL, > > + sa | DCRO_PEGPL_OMRxMSKL_VAL); > > Didn't you just change "sa | 3" to "sa | 1" ? > Yes, but I think that is correct for it to be "1". The data sheets for these parts that I checked had bit 1 marked as reserved. Only OMR1MSKL and OMR3MSKL had extra definitions such as the _IO and _UOT. The parts I checked which were the sheets for the EX and SX (which cover another 6 or 7 parts) all had it with just a single bit defined on that register. Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 1/1] powerpc/4xx: enable and fix pcie gen1/gen2 on the 460sx
Thanks Tony, some comments below.
On Thu, Jul 14, 2011 at 11:16:27AM +1000, Tony Breeds wrote:
>
> > +static void __init ppc460sx_pciex_check_link(struct ppc4xx_pciex_port
> > *port)
> > +{
> > + void __iomem *mbase;
> > + int attempt = 50;
> > +
> > + port->link = 0;
> > +
> > + mbase = ioremap(port->cfg_space.start + 0x, 0x1000);
>
> Why + 0x ? ppc4xx_pciex_port_setup_hose() does:
> mbase = ioremap(port->cfg_space.start + 0x1000, 0x1000);
> so isn't one of these statements is wrong?
yes, that doesn't look right. I'll verify that and make
sure that it works correctly and resubmit the patch.
>
> > + if (mbase == NULL) {
> > + printk(KERN_ERR "%s: Can't map internal config space !",
> > + port->node->full_name);
> > + goto done;
> > + }
> > +
> > + while (attempt && (0 == (in_le32(mbase + PECFG_460SX_DLLSTA)
> > + & 0x0010))) {
>
> Nitpicking, I think it'd be nice if there was #define for 0x0010
> perhaps: #define PECFG_460SX_DLLSTA_LINKUP 0x0010
ok.
> >
> > - if (ppc4xx_pciex_hwops->check_link)
> > - ppc4xx_pciex_hwops->check_link(port);
> > -
> > /*
> > * Initialize mapping: disable all regions and configure
> > * CFG and REG regions based on resources in the device tree
> > */
> > ppc4xx_pciex_port_init_mapping(port);
> >
> > + if (ppc4xx_pciex_hwops->check_link)
> > + ppc4xx_pciex_hwops->check_link(port);
> > +
>
> Why move this? You already iorempat the cfg space.
This was what I was asking about before. The reason that I
swapped the order of the init_mapping and check_link is
because the init_mapping currently sets up the cfgbax
registers. Those setup the base address of the
configuration space on the PLB side of the bus. As far as I
could determine, I cannot access the config space until
those registers are configured. I need to touch the config
space in order to do the check_link b/c the 460sx uses the
extended config space to keep track of the link status. I
looked at init mapping and based on what it did I did not
see any potential adverse effects.
> > out_le32(mbase + PECFG_POM2LAH, pciah);
> > @@ -1591,8 +1632,7 @@ static int __init ppc4xx_setup_one_pciex_POM(struct
> > ppc4xx_pciex_port *port,
> > dcr_write(port->dcrs, DCRO_PEGPL_OMR3BAH, lah);
> > dcr_write(port->dcrs, DCRO_PEGPL_OMR3BAL, lal);
> > dcr_write(port->dcrs, DCRO_PEGPL_OMR3MSKH, 0x7fff);
> > - /* Note that 3 here means enabled | IO space !!! */
> > - dcr_write(port->dcrs, DCRO_PEGPL_OMR3MSKL, sa | 3);
> > + dcr_write(port->dcrs, DCRO_PEGPL_OMR3MSKL, sa);
> > break;
> > }
>
> I think you really want to check the definitions for OMRs 2 and 3 to verify
> that this is right.
Thanks, good catch. I'll change the first case block to
include a switch on the 460sx. The first case statement
needs to be | 0x5, while the others need to stay 0x3.
Ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
[PATCH 1/1] powerpc/4xx: enable and fix pcie gen1/gen2 on the 460sx
Adds a register to the config space for the 460sx. Changes the vc0
detect to a pll detect. maps configuration space to test the link
status. changes the setup to enable gen2 devices to operate at gen2
speeds. fixes mapping that was not correct for the 460sx.
tested on the 460sx eiger and custom board
Signed-off-by: Ayman El-Khashab
---
arch/powerpc/sysdev/ppc4xx_pci.c | 83 +
arch/powerpc/sysdev/ppc4xx_pci.h |3 +
2 files changed, 68 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/sysdev/ppc4xx_pci.c b/arch/powerpc/sysdev/ppc4xx_pci.c
index ad330fe..273963b 100644
--- a/arch/powerpc/sysdev/ppc4xx_pci.c
+++ b/arch/powerpc/sysdev/ppc4xx_pci.c
@@ -1092,6 +1092,10 @@ static int __init ppc460sx_pciex_core_init(struct
device_node *np)
mtdcri(SDR0, PESDR1_460SX_HSSSLEW, 0x);
mtdcri(SDR0, PESDR2_460SX_HSSSLEW, 0x);
+ /* Set HSS PRBS enabled */
+ mtdcri(SDR0, PESDR0_460SX_HSSCTLSET, 0x1130);
+ mtdcri(SDR0, PESDR2_460SX_HSSCTLSET, 0x1130);
+
udelay(100);
/* De-assert PLLRESET */
@@ -1132,9 +1136,6 @@ static int ppc460sx_pciex_init_port_hw(struct
ppc4xx_pciex_port *port)
dcri_clrset(SDR0, port->sdr_base + PESDRn_UTLSET2,
0, 0x0100);
- /*Gen-1*/
- mtdcri(SDR0, port->sdr_base + PESDRn_460SX_RCEI, 0x0800);
-
dcri_clrset(SDR0, port->sdr_base + PESDRn_RCSSET,
(PESDRx_RCSSET_RSTGU | PESDRx_RCSSET_RSTDL),
PESDRx_RCSSET_RSTPYN);
@@ -1148,14 +1149,42 @@ static int ppc460sx_pciex_init_utl(struct
ppc4xx_pciex_port *port)
{
/* Max 128 Bytes */
out_be32 (port->utl_base + PEUTL_PBBSZ, 0x);
+ /* Assert VRB and TXE - per datasheet turn off addr validation */
+ out_be32(port->utl_base + PEUTL_PCTL, 0x8080);
return 0;
}
+static void __init ppc460sx_pciex_check_link(struct ppc4xx_pciex_port *port)
+{
+ void __iomem *mbase;
+ int attempt = 50;
+
+ port->link = 0;
+
+ mbase = ioremap(port->cfg_space.start + 0x, 0x1000);
+ if (mbase == NULL) {
+ printk(KERN_ERR "%s: Can't map internal config space !",
+ port->node->full_name);
+ goto done;
+ }
+
+ while (attempt && (0 == (in_le32(mbase + PECFG_460SX_DLLSTA)
+ & 0x0010))) {
+ attempt--;
+ mdelay(10);
+ }
+ if (attempt)
+ port->link = 1;
+done:
+ iounmap(mbase);
+
+}
+
static struct ppc4xx_pciex_hwops ppc460sx_pcie_hwops __initdata = {
.core_init = ppc460sx_pciex_core_init,
.port_init_hw = ppc460sx_pciex_init_port_hw,
.setup_utl = ppc460sx_pciex_init_utl,
- .check_link = ppc4xx_pciex_check_link_sdr,
+ .check_link = ppc460sx_pciex_check_link,
};
#endif /* CONFIG_44x */
@@ -1338,15 +1367,15 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
if (rc != 0)
return rc;
- if (ppc4xx_pciex_hwops->check_link)
- ppc4xx_pciex_hwops->check_link(port);
-
/*
* Initialize mapping: disable all regions and configure
* CFG and REG regions based on resources in the device tree
*/
ppc4xx_pciex_port_init_mapping(port);
+ if (ppc4xx_pciex_hwops->check_link)
+ ppc4xx_pciex_hwops->check_link(port);
+
/*
* Map UTL
*/
@@ -1360,13 +1389,23 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
ppc4xx_pciex_hwops->setup_utl(port);
/*
-* Check for VC0 active and assert RDY.
+* Check for VC0 active or PLL Locked and assert RDY.
*/
if (port->sdr_base) {
- if (port->link &&
- ppc4xx_pciex_wait_on_sdr(port, PESDRn_RCSSTS,
-1 << 16, 1 << 16, 5000)) {
- printk(KERN_INFO "PCIE%d: VC0 not active\n",
port->index);
+ if (of_device_is_compatible(port->node,
+ "ibm,plb-pciex-460sx")){
+ if (port->link && ppc4xx_pciex_wait_on_sdr(port,
+ PESDRn_RCSSTS,
+ 1 << 12, 1 << 12, 5000)) {
+ printk(KERN_INFO "PCIE%d: PLL not locked\n",
+ port->index);
+ port->link = 0;
+ }
+ } else if (port->link &&
+ ppc4xx_pciex_wait_on_sdr(port, PESDRn_RCSSTS,
+
[PATCH 0/1] powerpc/4xx: enable and fix pcie gen1/gen2 on the 460sx
Old code didn't check the proper registers for the 460sx and so the link check would always fail. This changes the link check to use the config space reg and adds the definition for the register. Also changes the vc0 to a pll check. Sets up the port for gen2 speeds. This uses the check_link function to specialize the link detect for the 460sx. Tested on an eiger and custom board. Note that is swaps the order of the link check and dcr initialization since the config space needs the DCRs setup before it can be mapped and used to check the link. Ayman El-Khashab (1): powerpc/4xx: enable and fix pcie gen1/gen2 on the 460sx arch/powerpc/sysdev/ppc4xx_pci.c | 83 + arch/powerpc/sysdev/ppc4xx_pci.h |3 + 2 files changed, 68 insertions(+), 18 deletions(-) -- 1.7.4.3 ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
grasping why mpt2sas fails with BITS_PER_LONG conditional
I've had problems with the mpt2sas driver (drivers/scsi/mpt2sas/mpt2sas_base.c) when using it on my ppc44x. Basically, there is a check in the code for BITS_PER_LONG and certain code is executed if it is 64 vs 32 bit. I am guessing it is a problem with the driver but it only shows up on powerpc. If i force it to take the 64 bit path it works fine, but if I let the preprocessor decide it will take the 32 bit path and fail later. I am not configuring my kernel as 64 bit, but I am guessing this has to do with the 36 bit plb bus on these parts. can somebody enlighten me as to what the right thing to do here is? thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH] 4xx: Add check_link to struct ppc4xx_pciex_hwops
On Tue, Jul 12, 2011 at 02:04:04PM -0400, Josh Boyer wrote:
> On Tue, Jul 12, 2011 at 12:40:07PM -0500, Ayman El-Khashab wrote:
> >On Fri, Jul 01, 2011 at 04:44:24PM +1000, Tony Breeds wrote:
> >> All current pcie controllers unconditionally use SDR to check the link and
> >> poll for reset.
> >
> >I was able to apply this patch and then modify the 460SX to
> >work correctly, so I think it is fine. There is only 1
> >comment below. So how does one supply a patch atop another
> >patch?
> >
> >Best,
> >Ayman
> >
> >> +static int __init ppc4xx_pciex_port_reset_sdr(struct ppc4xx_pciex_port
> >> *port)
> >> +{
> >> + printk(KERN_INFO "PCIE%d: Checking link...\n",
> >> + port->index);
> >
> >Its not a functional problem, but this printk belongs in the
> >check link if anywhere rather than the reset.
>
> I've got this queued in my tree locally. I can make that change before
> I push it out.
>
Ok, so let me ask the following ... will it cause trouble if
I swap the sequence of the calls to the following in xxx_port_init
ppc4xx_pciex_port_init_mapping(...)
and
if (ppc4xx_pciex_hwops->check_link)...
The reason is that at least on the 460SX, the link check is
done via registers in the config space. But the init_mapping is
needed to setup some of the DCRs to make the config space work.
In my check_link, i map the config space do the link checks
and then unmap since a superset of the space could be mapped
later.
Thanks,
ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH] 4xx: Add check_link to struct ppc4xx_pciex_hwops
On Fri, Jul 01, 2011 at 04:44:24PM +1000, Tony Breeds wrote:
> All current pcie controllers unconditionally use SDR to check the link and
> poll for reset.
I was able to apply this patch and then modify the 460SX to
work correctly, so I think it is fine. There is only 1
comment below. So how does one supply a patch atop another
patch?
Best,
Ayman
> +static int __init ppc4xx_pciex_port_reset_sdr(struct ppc4xx_pciex_port *port)
> +{
> + printk(KERN_INFO "PCIE%d: Checking link...\n",
> +port->index);
Its not a functional problem, but this printk belongs in the
check link if anywhere rather than the reset.
> +
> + /* Wait for reset to complete */
> + if (ppc4xx_pciex_wait_on_sdr(port, PESDRn_RCSSTS, 1 << 20, 0, 10)) {
> + printk(KERN_WARNING "PCIE%d: PGRST failed\n",
> +port->index);
> + return -1;
> + }
> + return 0;
> +}
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: powerpc/4xx: Regression failed on sil24 (and other) drivers
On Wed, Jun 29, 2011 at 11:42:03AM +1000, Benjamin Herrenschmidt wrote: > On Mon, 2011-06-27 at 06:31 -0500, Ayman El-Khashab wrote: > > On Mon, Jun 27, 2011 at 08:19:56PM +1000, Benjamin Herrenschmidt wrote: > > > On Sat, 2011-06-25 at 18:52 -0500, Ayman El-Khashab wrote: > > > > I noticed during a recent development with the 460SX that a > > > > simple device that once worked stopped. I did a bisect to > > > > find the offending commit and it turns out to be this one: > > > > > > > > 0e52247a2ed1f211f0c4f682dc999610a368903f is the first bad > > > > commit > > > > commit 0e52247a2ed1f211f0c4f682dc999610a368903f > > > > Author: Cam Macdonell > > > > Date: Tue Sep 7 17:25:20 2010 -0700 > > > > > > > > PCI: fix pci_resource_alignment prototype > > > > > > > I suspect you don't have CONFIG_PCI_QUIRKS enabled... I think that's the > cause of your problem. > > It looks like this config option controls both compiling the "generic" > quirks in from drivers/pci/quirk.c, and the actually mechanism for > having quirks in the first place (pci_fixup_device() goes away without > that config option). > > I think we probably want to unconditionally select that if CONFIG_PCI is > enabled in arch/powerpc... > > Can you try changing it and tell us if that helps ? Yes, that fixed our problem, thanks for your time. I am going to try to get the MSI to work. Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH v4]PPC4xx: Adding PCI(E) MSI support
On Wed, Jun 29, 2011 at 09:15:28AM +1000, Benjamin Herrenschmidt wrote:
> On Tue, 2011-06-28 at 17:31 -0500, Ayman El-Khashab wrote:
> > > > +static int ppc4xx_setup_pcieh_hw(struct platform_device *dev,
> > > > +struct resource res, struct
> > ppc4xx_msi *msi)
> > > > +{
> > > > +
> >
> >
> >
> > > > +
> > > > + msi->msi_dev = of_find_node_by_name(NULL, "ppc4xx-msi");
> > > > + if (msi->msi_dev)
> > > > + return -ENODEV;
> >
> > This does not look correct. I guess it should probably read
> >
> > if (!msi->msi_dev) .
>
> Indeed, that looks bogus. Rupjyoti, please test and send fixes if
> necessary, obviously this code has not been tested.
>
> This is not part of the bits I fixed up so I looks to me like the
> original patch was wrong (and thus obviously untested !!!)
>
Looking back through the mailing list, there have been
various incarnations of this patch to add MSI support to the
44x. Every one that I looked at had this same line of code
in it so I am not sure they worked. In any case I am trying
to make it work on my system (which is how I found the bug).
When I enable the "sdr-base" line in the MSI section of my
dts, it just reboots continuosly right after "Loading Device
Tree ". I tried renaming it to "msi-sdr-base" just in
case there was a conflict (since it is reading through the
entire tree) but that did not help. If I understand
correctly, the ppc4xx_msi_probe function must be executing
very early since I suspect something in setup_pcieh_hw is
what causes it to fail.
ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH v4]PPC4xx: Adding PCI(E) MSI support
On Thu, May 26, 2011 at 03:24:44PM +1000, Benjamin Herrenschmidt wrote:
>
> Please check the result and send any "fixup" patch that might be
> necessary.
>
> > +static int ppc4xx_setup_pcieh_hw(struct platform_device *dev,
> > +struct resource res, struct ppc4xx_msi *msi)
> > +{
> > +
> > +
> > + msi->msi_dev = of_find_node_by_name(NULL, "ppc4xx-msi");
> > + if (msi->msi_dev)
> > + return -ENODEV;
This does not look correct. I guess it should probably read
if (!msi->msi_dev) .
Ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: powerpc/4xx: Regression failed on sil24 (and other) drivers
On Mon, Jun 27, 2011 at 08:19:56PM +1000, Benjamin Herrenschmidt wrote: > On Sat, 2011-06-25 at 18:52 -0500, Ayman El-Khashab wrote: > > I noticed during a recent development with the 460SX that a > > simple device that once worked stopped. I did a bisect to > > find the offending commit and it turns out to be this one: > > > > 0e52247a2ed1f211f0c4f682dc999610a368903f is the first bad > > commit > > commit 0e52247a2ed1f211f0c4f682dc999610a368903f > > Author: Cam Macdonell > > Date: Tue Sep 7 17:25:20 2010 -0700 > > > > PCI: fix pci_resource_alignment prototype > > > > > > The device driver fails with "error -22" on a 460SX (which > > has the 36 bit pci space). > > > > sil24 /drivers/ata/sata_sil24.c > > Can you send a dmesg & output of /proc/iomem & ioport with and without > the patch (same kernel otherwise) ? > > Also can you try to figure out (printk's) where in the driver does it > fail ? (Which function fails) Yes, here is the output from a canyonlands (460ex) that exhibits the same problem and in the same place. Of the two devices I have that fail (sil24 and one other), both fail in exactly the same place in lib/devres.c within the function pcim_iomap_regions. In that function, there is the following call -- it fails b/c len returns 0 and tha failure bubbles up to "error -22". len = pci_resource_len(pdev, i); > It's possible that this changes something in the core resource > assignment code causing something else to fail elsewhere or exposing > another bug elsewhere with the consequence of leaving the SiL with badly > assigned resources. That was my initial thought as well, but I wasn't versed enough in the pci magic in order to completely figure it out. Here is the output, it is dmesg, iomem, then ioports for the passing and then the failing cases. thanks ayman == Passing == Using PowerPC 44x Platform machine description Linux version 2.6.36-rc3-00186-g0e52247-dirty (aymane@lablinux) (gcc version 4.2.2) #18 Sat Jun 25 13:51:44 CDT 2011 Found initrd at 0xdfa5c000:0xdfe4cbfa Found legacy serial port 0 for /plb/opb/serial@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=6451612, speed=0 Found legacy serial port 1 for /plb/opb/serial@ef600400 mem=4ef600400, taddr=4ef600400, irq=0, clk=6451612, speed=0 Top of RAM: 0x2000, Total RAM: 0x2000 Memory hole size: 0MB Zone PFN ranges: DMA 0x -> 0x0002 Normal empty HighMem empty Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x -> 0x0002 On node 0 totalpages: 131072 free_area_init_node: node 0, pgdat c03b9f48, node_mem_map c03ed000 DMA zone: 1024 pages used for memmap DMA zone: 0 pages reserved DMA zone: 130048 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Kernel command line: root=/dev/ram rw mem=512M ip=169.254.0.180:169.254.0.100:169.254.0.100:255.255.255.0:tanosx:eth0:off panic=1 console=ttyS0,57600 PID hash table entries: 2048 (order: 1, 8192 bytes) Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) High memory: 0k Memory: 511668k/524288k available (3692k kernel code, 12620k reserved, 176k data, 141k bss, 184k init) Kernel virtual memory layout: * 0xfffcf000..0xf000 : fixmap * 0xffc0..0xffe0 : highmem PTEs * 0xffa0..0xffc0 : consistent mem * 0xffa0..0xffa0 : early ioremap * 0xe100..0xffa0 : vmalloc & ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 NR_IRQS:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 800.10 MHz time_init: processor frequency = 800.10 MHz clocksource: timebase mult[50] shift[22] registered clockevent: decrementer mult[ccf7] shift[32] cpu[0] pid_max: default: 32768 minimum: 301 Mount-cache hash table entries: 512 NET: Registered protocol family 16 i2c-core: driver [dummy] registered irq: irq 11 on host /interrupt-controller1 mapped to virtual irq 18 256k L2-cache enabled PCIE0: Checking link... PCIE0: No device detected. PCI host bridge /plb/pciex@d (primary) ranges: MEM 0x000e..0x000e7fff -> 0x8000 MEM 0x000f..0x000f000f -> 0x IO 0x000f8000..0x000f8000 -> 0x Removing ISA
Re: [PATCH 0/1] ppc4xx: Fix PCIe scanning for the 460SX
On Tue, May 31, 2011 at 02:27:19PM -0700, Tirumala Marri wrote: > Not sure how I would know -- But with my eiger kit, I got a > cd from amcc that had a patched 2.6.30 or something kernel > in it to support the 460SX. The pci code was basically > subverted by adding a "port->link=1" at the very end of the > link check to always force it to succeed. However this code > never appeared in the public git repositories, so I am not > sure how the 460SX functioned (if it ever did) since the > link check that is in there now can't work on that SOC. > > > [marri]I don't know what PCI-E devices you use. We use E1000 > As a device for testing. Link-up was working before the submission. > There were some changes happened afterwards to the common code > Which seems to affected all 4xx devices. I will try latest code > On our other SoCs. Well, briefly we build a custom processor board that has a couple powerpcs on it and some pcie switches. We end up plugging our devices into the switches which is unlike what most people try. In any case, we use several different drive controllers for SAS, SCSI, and SSDs. For testing purposes (i.e. when things don't go the way they should) we have an adapter to make "regular" pcie slots that we plug in whatever. But our switches are on the board with the CPUs so as far as the CPU goes, it will always see a gen-2 link 8 lane link, irrespective of what the endpoint actually is. We let the switches handle the speed changes and the port bifurcation. > I took care of that in my patch. Basically it let the > system go to gen-2 speeds and negotiate down. > [marri] Great thx. Ok, so I am back from doing whatever it is that I do. Shall I go ahead and take a stab at making a new functor to handle the "link check"? -- which IIRC was the thought from last month about how best to handle the 460SX and whatever some people were using internally. Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
powerpc/4xx: Regression failed on sil24 (and other) drivers
I noticed during a recent development with the 460SX that a simple device that once worked stopped. I did a bisect to find the offending commit and it turns out to be this one: 0e52247a2ed1f211f0c4f682dc999610a368903f is the first bad commit commit 0e52247a2ed1f211f0c4f682dc999610a368903f Author: Cam Macdonell Date: Tue Sep 7 17:25:20 2010 -0700 PCI: fix pci_resource_alignment prototype I found it working with 2.6.36 but it seems that it is in the current trunk as well. I patched my code to take out this commit and (quickly) verified it was ok. I am guessing the patch is ok since it converts int types to resource_size_t. My guess is that the problem is in the sil24 driver but I did not see anything obvious in that code. Any tips on what could be wrong? Is the problem potentially somewhere being called by that code? The device driver fails with "error -22" on a 460SX (which has the 36 bit pci space). sil24 /drivers/ata/sata_sil24.c Thanks Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 0/1] ppc4xx: Fix PCIe scanning for the 460SX
On Tue, May 24, 2011 at 01:40:12PM +1000, Benjamin Herrenschmidt wrote: > On Thu, 2011-05-12 at 11:16 -0700, Tirumala Marri wrote: > > The register above > > PECFGn_DLLSTA is actually in the PCIe configuration space so > > we would have to map that in to be able to read that > > register during the link check. Is that correct or ok? > > [marri] yes, you need to program DCR register access these local PCIE_CFG > > registers. > > We do I think, tho we might have to re-order some stuff. I'm facing a > similar issue with some internal design here, I'm thinking off adding a > new hook to the ppc4xx_pciex_hwops for link checking to replace the SDR > business. That would probably solve the existing problem. I might be remembering it wrong (or reading it wrong) or both. But I thought that the config space was mapped in the setup_hose that happens after (and if) the link check finished successfully. > The interesting question of course is whether that 460SX stuff is the > same as what we're using internally :-) Not sure how I would know -- But with my eiger kit, I got a cd from amcc that had a patched 2.6.30 or something kernel in it to support the 460SX. The pci code was basically subverted by adding a "port->link=1" at the very end of the link check to always force it to succeed. However this code never appeared in the public git repositories, so I am not sure how the 460SX functioned (if it ever did) since the link check that is in there now can't work on that SOC. > > Lastly, what was the reason for forcing the original code to > > be GEN-1 speeds? > > [marri] Gen-2 need some extra checks compared to Gen-1. > > There were not many Gen-2 devices at the time of submission > > To test them. > > Can we fix that ? > I took care of that in my patch. Basically it let the system go to gen-2 speeds and negotiate down. Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 0/1] ppc4xx: Fix PCIe scanning for the 460SX
On Thu, May 05, 2011 at 09:44:27AM -0700, Tirumala Marri wrote: > > > > Also, the patch removes the code for waiting for the link to be up with > > a comment "What DCR has the link status on the 460SX?". Please fix that > > (Tirumala, can you provide the missing information ?) > > > > It is not one register. Here is the flow for Gen-1. > 1. PECFGn_DLLSTA[3] will be asserted when pci-e link comes up. > 2. now progream the UTL buffer configuration registers. > 3. SW should assert PEUTLn_PCTL[0] to cause flow control initialization. > This is memory mapped using GPL register REGBH , REBAL and REGMSK > 4. Now check for the PEUTLN_PSTA[8] for the flow control init completion. So what is the best way to handle this? It appears (based on the comments of others and my own experience) that there is no DCR that exists and behaves the way that previous SOCs behaved to give us the link status? The register above PECFGn_DLLSTA is actually in the PCIe configuration space so we would have to map that in to be able to read that register during the link check. Is that correct or ok? I've communicated with some people over email and they had tried the (PESDRn_HSSLySTS) register. Recognizing that there exists one of these for each port/lane, is there a way to use this one? It is in the indirect DCR space. I'd tried this myself and never did get it to do anything but I could have been looking at the wrong lane or something. Lastly, what was the reason for forcing the original code to be GEN-1 speeds? ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 0/1] ppc4xx: Fix PCIe scanning for the 460SX
On Tue, Apr 12, 2011 at 07:09:49PM -0700, Tirumala Marri wrote: > You originally submitted the support for 460ex. Can you chime in (and > review Ayman patch) please ? > > [Marri] Ben sure I will review it and send you my feedback in couple of > days. Is there any update on this patch? Any comment? Any reason it _cant_ be included? Thanks Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
[PATCH 0/1] ppc4xx: Fix PCIe scanning for the 460SX
From: Ayman El-Khashab This patch is to fix the PCIe on the 460SX CPU. As far as I can tell, the 460SX must be using a different core than the previous 4xx SOCs. The registers aren't the same and it appears DCRs that existed on previous parts don't exist on this one. Perhaps somebody from AMCC can chime in. In any case, the main problem was that the TX and RX weren't enabled so nothing downstream from the RP was ever found. And the check to set "port->link" would never work (as far as I could tell). I could not find any DCRs in this part to indicate when the link was up and there isn't an obvious loopback, so the following call would fail and result in no devices found. ppc4xx_pciex_wait_on_sdr(port, PESDRn_LOOP, 1 << 28, 1 << 28, 100) It would be better to detect the link status, but I don't see an obvious way to do that from the data sheet. U-boot just skips the link check as well for this part. Ayman El-Khashab (1): ppc4xx: Fix PCIe scanning for the 460SX arch/powerpc/sysdev/ppc4xx_pci.c | 35 +++ 1 files changed, 31 insertions(+), 4 deletions(-) -- 1.7.4.3 ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
[PATCH 1/1] ppc4xx: Fix PCIe scanning for the 460SX
From: Ayman El-Khashab
The 460SX uses a different register set than previous 44x PCIe CPUs,
so some of the checks were not valid. Added an enable for the TX
and RX. For the 460SX only: Bypassed VCO check and added PLL check.
Bypassed the link check. Changed to advertise gen 2 speeds.
Signed-off-by: Ayman El-Khashab
---
arch/powerpc/sysdev/ppc4xx_pci.c | 35 +++
1 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/sysdev/ppc4xx_pci.c b/arch/powerpc/sysdev/ppc4xx_pci.c
index 156aa7d..1c854a7 100644
--- a/arch/powerpc/sysdev/ppc4xx_pci.c
+++ b/arch/powerpc/sysdev/ppc4xx_pci.c
@@ -1058,6 +1058,8 @@ static int __init ppc460sx_pciex_core_init(struct
device_node *np)
static int ppc460sx_pciex_init_port_hw(struct ppc4xx_pciex_port *port)
{
+ /* 500ms should be long enough */
+ int timeout = 5;
if (port->endpoint)
dcri_clrset(SDR0, port->sdr_base + PESDRn_UTLSET2,
@@ -1066,15 +1068,20 @@ static int ppc460sx_pciex_init_port_hw(struct
ppc4xx_pciex_port *port)
dcri_clrset(SDR0, port->sdr_base + PESDRn_UTLSET2,
0, 0x0100);
- /*Gen-1*/
- mtdcri(SDR0, port->sdr_base + PESDRn_460SX_RCEI, 0x0800);
dcri_clrset(SDR0, port->sdr_base + PESDRn_RCSSET,
(PESDRx_RCSSET_RSTGU | PESDRx_RCSSET_RSTDL),
PESDRx_RCSSET_RSTPYN);
- port->has_ibpre = 1;
+ /* Poll for the PHY reset */
+ while (timeout-- && !(mfdcr(port->sdr_base + PESDRn_RCSSTS) & 0x1))
+ udelay(10);
+
+ if (!timeout)
+ return -1;
+
+ port->has_ibpre = 1;
return 0;
}
@@ -1082,6 +1089,8 @@ static int ppc460sx_pciex_init_utl(struct
ppc4xx_pciex_port *port)
{
/* Max 128 Bytes */
out_be32 (port->utl_base + PEUTL_PBBSZ, 0x);
+ out_be32 (port->utl_base + PEUTL_PCTL,0x8080);
+
return 0;
}
@@ -1308,6 +1317,7 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
* config space accesses. That way, it will be easier to implement
* hotplug later on.
*/
+#ifndef CONFIG_460SX
if (!port->has_ibpre ||
!ppc4xx_pciex_wait_on_sdr(port, PESDRn_LOOP,
1 << 28, 1 << 28, 100)) {
@@ -1325,6 +1335,10 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
}
} else
printk(KERN_INFO "PCIE%d: No device detected.\n", port->index);
+#else
+ /* XXX What DCR has the link status on the 460SX? */
+ port->link = 1;
+#endif
/*
* Initialize mapping: disable all regions and configure
@@ -1344,8 +1358,9 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
if (ppc4xx_pciex_hwops->setup_utl)
ppc4xx_pciex_hwops->setup_utl(port);
+#ifndef CONFIG_460SX
/*
-* Check for VC0 active and assert RDY.
+* Check for VC0 active
*/
if (port->link &&
ppc4xx_pciex_wait_on_sdr(port, PESDRn_RCSSTS,
@@ -1353,7 +1368,19 @@ static int __init ppc4xx_pciex_port_init(struct
ppc4xx_pciex_port *port)
printk(KERN_INFO "PCIE%d: VC0 not active\n", port->index);
port->link = 0;
}
+#else
+ /*
+* Check for PLL Locked
+*/
+ if (port->link &&
+ ppc4xx_pciex_wait_on_sdr(port, PESDRn_RCSSTS,
+1 << 12, 1 << 12, 5000)) {
+ printk(KERN_INFO "PCIE%d: PLL not locked\n", port->index);
+ port->link = 0;
+ }
+#endif
+ /* Assert RDY */
dcri_clrset(SDR0, port->sdr_base + PESDRn_RCSSET, 0, 1 << 20);
msleep(100);
--
1.7.4.3
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: problem PCIe LSI detected at 32 device addresses (ppc460ex)
On Sun, Apr 03, 2011 at 04:09:26PM -0600, Grant Likely wrote:
> On Sun, Apr 3, 2011 at 3:52 PM, Benjamin Herrenschmidt
> wrote:
> >
> >> Ok, I've narrowed the scope of the problem some. ?I moved forward
> >> to a more recent kernel (2.6.31 to 2.6.36) and that resolved the
> >> problem of the controller showing up as every device on the bus.
> >> However, from 2.6.37 to the current HEAD, I have not been able to
> >> build a kernel to run on the 460EX. ?I tried 2.6.37, 2.6.38, and
> >> the HEAD and all result in the following kernel panic. ?I am not
> >> sure how to proceed here. ?I suppose we can stick with 2.6.36 since
> >> it works, but I'd like to understand what it might take to remedy
> >> this.
> >
> > Smells like somebody changed something with the OF flash code... Josh,
> > Grant, any idea what's up there ?
>
> Not sure, more information would be helpful.
>
> Ayman, if you do a 'git log v2.6.36.. drivers/mtd/maps/physmap_of.c',
> then you'll see a list of commits touching the mtd driver. Would you
> be able to do a 'git checkout ' on each of those are report
> back on at what point things stop working? Actually, a full bisect
> between 2.6.36 and 2.6.37 would be best, but this is a good start if
> you're limited on time. Once you find the first commit where it
> fails, do a 'git checkout ~1' to confirm that it is in fact the
> commit that causes the breakage.
I can try to find the commit tomorrow. In the interim, i've pasted
the dts below. The board was originally based on the canyonlands, but
we've made some changes, mostly to the pcie. we run the 1-l port in
endpoint mode, we have a chain of plx switches and devices on the 4-l
port. One item that I don't think would matter, but is not too common
is that we are booting these over the pci bus from another PPCs memory.
I only mention this since this failure is during boot, though everything
should by local to the cpu by this time.
>
> Can you also post your device tree please?
Here is the device tree for our custom board.
/dts-v1/;
/ {
#address-cells = <2>;
#size-cells = <1>;
model = "amcc,canyonlands";
compatible = "amcc,canyonlands";
dcr-parent = <&{/cpus/cpu@0}>;
aliases {
ethernet0 = &EMAC0;
ethernet1 = &EMAC1;
serial0 = &UART0;
serial1 = &UART1;
};
cpus {
#address-cells = <1>;
#size-cells = <0>;
cpu@0 {
device_type = "cpu";
model = "PowerPC,460EX";
reg = <0x>;
clock-frequency = <0>; /* Filled in by U-Boot */
timebase-frequency = <0>; /* Filled in by U-Boot */
i-cache-line-size = <32>;
d-cache-line-size = <32>;
i-cache-size = <32768>;
d-cache-size = <32768>;
dcr-controller;
dcr-access-method = "native";
};
};
memory {
device_type = "memory";
reg = <0x 0x 0x>; /* Filled in by
U-Boot */
};
UIC0: interrupt-controller0 {
compatible = "ibm,uic-460ex","ibm,uic";
interrupt-controller;
cell-index = <0>;
dcr-reg = <0x0c0 0x009>;
#address-cells = <0>;
#size-cells = <0>;
#interrupt-cells = <2>;
};
UIC1: interrupt-controller1 {
compatible = "ibm,uic-460ex","ibm,uic";
interrupt-controller;
cell-index = <1>;
dcr-reg = <0x0d0 0x009>;
#address-cells = <0>;
#size-cells = <0>;
#interrupt-cells = <2>;
interrupts = <0x1e 0x4 0x1f 0x4>; /* cascade */
interrupt-parent = <&UIC0>;
};
UIC2: interrupt-controller2 {
compatible = "ibm,uic-460ex","ibm,uic";
interrupt-controller;
cell-index = <2>;
dcr-reg = <0x0e0 0x009>;
#address-cells = <0>;
#size-cells = <0>;
#interrupt-cells = <2>;
interrupts = <0xa 0x4 0xb 0x4>; /* cascade */
interrupt-parent = <&UIC0>;
};
UIC3: interrupt-controller3 {
compatible = "ibm,uic-460ex","ibm,uic";
interrupt-controller;
cell-index = <3>;
dcr-reg = <0x0f0 0x009>;
#address-cells = <0>;
#size-cells = <0>;
#interrupt-cells = <2>;
interrupts = <0x10 0x4 0x11 0x4>; /* cascade */
interrupt-parent = <&UIC0>;
};
SDR0: sdr {
compatible = "ibm,sdr-460ex";
Re: problem PCIe LSI detected at 32 device addresses (ppc460ex)
On Fri, Apr 01, 2011 at 11:26:19AM -0500, Ayman El-Khashab wrote: > I've got an LSI SAS2008 controller (w/ firmware v9) that works > fine in a Linux PC w/ a recent kernel. It does NOT work on my > 460EX board. What I find is that the device shows up as every > device on the subordinate bus where it is really located. (see > the dumps below). With the LSI firmware v8 this didn't occur > and it showed up correctly as a single device. I am not sure Ok, I've narrowed the scope of the problem some. I moved forward to a more recent kernel (2.6.31 to 2.6.36) and that resolved the problem of the controller showing up as every device on the bus. However, from 2.6.37 to the current HEAD, I have not been able to build a kernel to run on the 460EX. I tried 2.6.37, 2.6.38, and the HEAD and all result in the following kernel panic. I am not sure how to proceed here. I suppose we can stick with 2.6.36 since it works, but I'd like to understand what it might take to remedy this. Thanks U-Boot 2008.10 (Mar 15 2011 - 18:44:08) CPU: AMCC PowerPC 460EX Rev. A at 800 MHz (PLB=200, OPB=100, EBC=100 MHz) Security/Kasumi support Bootstrap Option D - Boot ROM Location PCI Internal PCI arbiter disabled 32 kB I-Cache 32 kB D-Cache Board: tanosx-slave - Tanisys SX Platform, 2*PCIeToggling PLX reset , Rev. 0 I2C: ready DRAM: Auto calibration |*** -- *** best_result window size: 231 *** best_result WRDTR: 0x0001 *** best_result CLKTR: 0x0001 *** best_result RQFD: 0x002e *** best_result RFFD: 0x0230 *** best_result RDCC: 0x4000 *** -- 512 MB (ECC enabled, 400 MHz, CL3) *** Warning - bad CRC, using default environment PCIE0: successfully set as endpoint PCIE1: successfully set as root-complex ### Unknown PB ### Net: No ethernet found. Type run flash_nfs to mount root filesystem over NFS Hit any key to stop autoboot: 0 ## Booting kernel from Legacy Image at ff00 ... Image Name: Linux-2.6.37-0-v2.6.37 Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size:2146637 Bytes = 2 MB Load Address: Entry Point: Verifying Checksum ... OK ## Loading init Ramdisk from Legacy Image at ff40 ... Image Name: Tanisys Ramdisk Image Image Type: PowerPC Linux RAMDisk Image (gzip compressed) Data Size:3830562 Bytes = 3.7 MB Load Address: Entry Point: Verifying Checksum ... OK ## Flattened Device Tree blob at ff3e Booting using the fdt blob at 0xff3e Uncompressing Kernel Image ... OK Loading Device Tree to 007fa000, end 007ffe96 ... OK Loading Ramdisk to 1fabe000, end 1fe65322 ... OK Using PowerPC 44x Platform machine description Linux version 2.6.37-0-v2.6.37 (aymane@lablinux) (gcc version 4.2.2) #16 Fri Apr 1 15:20:03 CDT 2011 Found initrd at 0xdfabe000:0xdfe65322 Zone PFN ranges: DMA 0x -> 0x0002 Normal empty HighMem empty Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x -> 0x0002 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Kernel command line: root=/dev/ram rw mem=512M ip=tanosx-slave:eth0:off panic=1 console=ttyS0,5 7600 PID hash table entries: 2048 (order: 1, 8192 bytes) Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Memory: 510452k/524288k available (4180k kernel code, 13836k reserved, 160k data, 1144k bss, 180k i nit) Kernel virtual memory layout: * 0xfffcf000..0xf000 : fixmap * 0xffc0..0xffe0 : highmem PTEs * 0xff20..0xffc0 : consistent mem * 0xff20..0xff20 : early ioremap * 0xe100..0xff20 : vmalloc & ioremap SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 NR_IRQS:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 UIC2 (32 IRQ sources) at DCR 0xe0 UIC3 (32 IRQ sources) at DCR 0xf0 clocksource: timebase mult[50] shift[22] registered pid_max: default: 4096 minimum: 301 Mount-cache hash table entries: 512 xor: measuring software checksum speed 8regs : 691.000 MB/sec 8regs_prefetch: 607.000 MB/sec 32regs: 882.000 MB/sec 32regs_prefetch: 748.000 MB/sec xor: using function: 32regs (882.000 MB/sec) NET: Registered protocol family 16 256k L2-cache enabled PCIE0: Checking link... PCIE0: Device detected, waiting for link... PCIE0: link is up ! PCI host bridge /plb/pciex@d (primary) ranges: MEM 0x000e..0x000e7fff -> 0x8000 IO 0x000f8000..0x000f8000 -> 0x 4xx PCI DMA offset set to 0x PCIE0: successfully set as endpoint PCIE1: Checking link... PCIE1: Device detected, waiting for link... PCIE1: link is up ! PCI host bridge /plb/pci
help debugging linux setting PLX Gen2 PCIe switches to gen1 speed
I've got a custom 460SX board with PLX gen2 (8616) switches. When u-boot comes up everything is fine and my switches are indicating they are in Gen2 PCIe mode. If I let it boot my linux kernel (denx 2.6.36.1), boot almost immediately, the LEDs on my switches start to blink indicating they have are now at Gen1 speed. I've turned on debug in ppc4xx_pci.c but don't see anything obvious. As far as when it happens, it looks like the last message to the console is "uncompressing kernel" and then the LEDs start blinking. seems that it indicates it is very early. I've also tried removing all the PCIe from the device tree, but that did not matter. In the data sheet it indicates that PEDSRn_RCEI can change the advertised rate. I've disabled that in the code, but it did not make any difference. I am just not sure where things are getting switched over. Any help would be appreciated. best ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: ppc44x - how do i optimize driver for tlb hits
On Sat, Sep 25, 2010 at 08:11:04AM +1000, Benjamin Herrenschmidt wrote: > On Fri, 2010-09-24 at 08:08 -0500, Ayman El-Khashab wrote: > > > > I suppose another option is to to use the kernel profiling option I > > always see but have never used. Is that a viable option to figure out > > what is happening here? > > With perf and stochastic sampling ? If you sample fast enough... but > you'll mostly point to your routine I suppose... though it might tell > you statistically where in your code, which -might- help. > Thanks I didn't end up profiling it b/c we found the biggest culprit. Basically we were mapping this memory in kernel space and as long as we did that ONLY everything was ok. But then we would mmap the physical addresses into user space. Using MAP_SHARED made it extremely slow. Using MAP_PRIVATE made it very fast. So it works, but why is MAP_SHARED that much slower? The other optimization was a change in the algorithm to take advantage of the L2 prefetching. Since we were operating on many simultaneous streams it seems that the cache performance was not good. thanks ame ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: ppc44x - how do i optimize driver for tlb hits
On Fri, Sep 24, 2010 at 06:30:34AM -0400, Josh Boyer wrote: > On Fri, Sep 24, 2010 at 02:43:52PM +1000, Benjamin Herrenschmidt wrote: > >> The DMA is what I use in the "real world case" to get data into and out > >> of these buffers. However, I can disable the DMA completely and do only > >> the kmalloc. In this case I still see the same poor performance. My > >> prefetching is part of my algo using the dcbt instructions. I know the > >> instructions are effective b/c without them the algo is much less > >> performant. So yes, my prefetches are explicit. > > > >Could be some "effect" of the cache structure, L2 cache, cache geometry > >(number of ways etc...). You might be able to alleviate that by changing > >the "stride" of your prefetch. My original theory was that it was having lots of cache misses. But since the algorithm works standalone fast and uses large enough buffers (4MB), much of the cache is flushed and replaced with my data. The cache is 32K, 8 way, 32b/line. I've crafted the algorithm to use those parameters. > > > >Unfortunately, I'm not familiar enough with the 440 micro architecture > >and its caches to be able to help you much here. > > Also, doesn't kmalloc have a limit to the size of the request it will > let you allocate? I know in the distant past you could allocate 128K > with kmalloc, and 2M with an explicit call to get_free_pages. Anything > larger than that had to use vmalloc. The limit might indeed be higher > now, but a 4MB kmalloc buffer sounds very large, given that it would be > contiguous pages. Two of them even less so. I thought so too, but at least in the current implementation we found empirically that we could kmalloc up to but no more than 4MB. We have also tried an approach in user memory and then using "get_user_pages" and building a scatter-gather. We found that the compare code doesn't perform any better. I suppose another option is to to use the kernel profiling option I always see but have never used. Is that a viable option to figure out what is happening here? ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: ppc44x - how do i optimize driver for tlb hits
On Fri, Sep 24, 2010 at 11:07:24AM +1000, Benjamin Herrenschmidt wrote: > On Thu, 2010-09-23 at 17:35 -0500, Ayman El-Khashab wrote: > > Anything you allocate with kmalloc() is going to be mapped by bolted > > > 256M TLB entries, so there should be no TLB misses happening in the > > > kernel case. > > > > > > > Hi Ben, can you or somebody elaborate? I saw the pinned tlb in > > 44x_mmu.c. > > Perhaps I don't understand the code fully, but it appears to map 256MB > > of "lowmem" into a pinned tlb. I am not sure what phys address lowmem > > means, but I assumed (possibly incorrectly) that it is 0-256MB. > > No. The first pinned entry (0...256M) is inserted by the asm code in > head_44x.S. The code in 44x_mmu.c will later map the rest of lowmem > (typically up to 768M but various settings can change that) using more > 256M entries. Thanks Ben, appreciate all your wisdom and insight. Ok, so my 460ex board has 512MB total, so how does that figure into the 768M? Is there some other heuristic that determines how these are mapped? > Basically, all of lowmem is permanently mapped with such entries. > > > When I get the physical addresses for my buffers after kmalloc, they > > all have addresses that are within my DRAM but start at about the > > 440MB mark. I end up passing those phys addresses to my DMA engine. > > Anything you get from kmalloc is going to come from lowmem, and thus be > covered by those bolted TLB entries. So is it reasonable to assume that everything on my system will come from pinned TLB entries? > > > When my compare runs it takes a huge amount of time in the assembly > > code doing memory fetches which makes me think that there are either > > tons of cache misses (despite the prefetching) or the entries have > > been purged > > What prefetching ? IE. The DMA operation -will- flush things out of the > cache due to the DMA being not cache coherent on 44x. The 440 also > doesn't have a working HW prefetch engine afaik (it should be disabled > in FW or early asm on 440 cores and fused out in HW on 460 cores afaik). > > So only explicit SW prefetching will help. > The DMA is what I use in the "real world case" to get data into and out of these buffers. However, I can disable the DMA completely and do only the kmalloc. In this case I still see the same poor performance. My prefetching is part of my algo using the dcbt instructions. I know the instructions are effective b/c without them the algo is much less performant. So yes, my prefetches are explicit. > > from the TLB and must be obtained again. As an experiment, I disabled > > my cache prefetch code and the algo took forever. Next I altered the > > asm to do the same amount of data but a smaller amount over and over > > so that less if fetched from main memory. That executed very quickly. > > >From that I drew the conclusion that the algorithm is memory > > bandwidth limited. > > I don't know what exactly is going on, maybe your prefetch stride isn't > right for the HW setup, or something like that. You can use xmon 'u' > command to look at the TLB content. Check that we have the 256M entries > mapping your data, they should be there. Ok, I will give that a try ... in addition, is there an easy way to use any sort of gprof like tool to see the system performance? What about looking at the 44x performance counters in some meaningful way? All the experiments point to the fetching being slower in the full program as opposed to the algo in a testbench, so I want to determine what it is that could cause that. thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: ppc44x - how do i optimize driver for tlb hits
On Fri, Sep 24, 2010 at 08:01:04AM +1000, Benjamin Herrenschmidt wrote: > On Thu, 2010-09-23 at 10:12 -0500, Ayman El-Khashab wrote: > > I've implemented a working driver on my 460EX. it allocates a couple > > of buffers of 4MB each. I have a custom memcmp algorithm in asm that > > is extremely fast in user space, but 1/2 as fast when run on these > > buffers. > > > > my tests are showing that the algorithm seems to be memory bandwidth > > bound. my guess is that i am having tlb or cache misses (my algo > > uses the dbct) that is slowing performance. curiously when in user > > space, i can affect the performance by small changes in the size of > > the buffer, i.e. 4MB + 32B is fast, 4MB + 4K is much worse. > > > > Can i adjust my driver code that is using kmalloc to make sure that > > the ppc44x has 4MB tlb entries for these and that they stay put? > > Anything you allocate with kmalloc() is going to be mapped by bolted > 256M TLB entries, so there should be no TLB misses happening in the > kernel case. > Hi Ben, can you or somebody elaborate? I saw the pinned tlb in 44x_mmu.c. Perhaps I don't understand the code fully, but it appears to map 256MB of "lowmem" into a pinned tlb. I am not sure what phys address lowmem means, but I assumed (possibly incorrectly) that it is 0-256MB. When I get the physical addresses for my buffers after kmalloc, they all have addresses that are within my DRAM but start at about the 440MB mark. I end up passing those phys addresses to my DMA engine. When my compare runs it takes a huge amount of time in the assembly code doing memory fetches which makes me think that there are either tons of cache misses (despite the prefetching) or the entries have been purged from the TLB and must be obtained again. As an experiment, I disabled my cache prefetch code and the algo took forever. Next I altered the asm to do the same amount of data but a smaller amount over and over so that less if fetched from main memory. That executed very quickly. >From that I drew the conclusion that the algorithm is memory bandwidth limited. In a standalone configuration (i.e. algorithm just using user memory, everything else identical), the speedup is 2-3x. So the limitation is not a hardware limit, it must be something that is happening when I execute the loads. (it is a compare algorithm, so it only does loads). Thanks Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
ppc44x - how do i optimize driver for tlb hits
I've implemented a working driver on my 460EX. it allocates a couple of buffers of 4MB each. I have a custom memcmp algorithm in asm that is extremely fast in user space, but 1/2 as fast when run on these buffers. my tests are showing that the algorithm seems to be memory bandwidth bound. my guess is that i am having tlb or cache misses (my algo uses the dbct) that is slowing performance. curiously when in user space, i can affect the performance by small changes in the size of the buffer, i.e. 4MB + 32B is fast, 4MB + 4K is much worse. Can i adjust my driver code that is using kmalloc to make sure that the ppc44x has 4MB tlb entries for these and that they stay put? thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Help with finding memory read performance problem
For our code we needed a fast memory compare of 5 buffers. I've implemented said routine in asm and it works fine and is very fast in the test bench. However when integrated with the app it is much less performant and we are trying to figure out why. The app in question gets the 5 4MB buffers in the kernel via kmalloc and then uses them for DMA. No other methods are being called for the memory besides kmalloc. This is an embedded system on the 460EX, so there is no drive, only RAM. Within the user code mmap is called on these buffers physical address and they are given to the compare routine. The result is slow. If I allocate buffers in user space then the performance is excellent. Next I implemented my compare routine within a kernel module so that it was using the kernel virtual addresses for each of the buffers. I did not see any change between this and the mmap approach. For comparison sake, using the kernel memory is about 19s whereas user memory is about 11s for the same size / configuration of buffer. In the test bench the algorithm is about 8s. The processor is not doing any other intensive tasks during these tests and the times are repeatable. Is something happening to mmap'd memory that causes the access to it to be slow? Is there a way to speed that up? Why are the kernel memory access slower than user memory? What is the best overall approach? Is it to DMA into user memory and then run the routines there? Is kmalloc not the best approach for kernel DMA memory? This is on linux 2.6.31.5 on 460EX thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
help with kernel panics in task swapper on 460ex
I have an odd problem when using the 460ex rev b processors. Previously, I'd used the rev A without any issues on the same pcbs. This happens on multiple units now. Basically, while running the system will just randomly kernel panic. We have seen this probably 4 or 5 times on a over the course of several days per board. On many occasions, the CPU will not kernel dump at all and will just be dead after having run for some amount of time. The kernel is 2.6.31.5 The exact details (though I don't think they are too useful are as follows. The one interesting item if I read it correctly, is that the CPU was trying to get instructions from c002 address. However, this board only has 512MB of memory so it seems that the address (if it is physical) isn't valid. Are there any tricks we can use to analyze this problem? Or is there any information we can collect to help pinpoint where this issue might lie? Thanks ame Unable to handle kernel paging request for data at address 0x8072f938 Faulting instruction address: 0xc002eac4 Oops: Kernel access of bad area, sig: 11 [#1] PowerPC 44x Platform Modules linked in: mapper NIP: c002eac4 LR: c004811c CTR: c000add0 REGS: c038be60 TRAP: 0300 Not tainted (2.6.31.5) MSR: 00021000 CR: 24644324 XER: DEAR: 8072f938, ESR: TASK = c036a318[0] 'swapper' THREAD: c038a000 GPR00: 00f6 c038bf10 c036a318 fffef700 fffef6f9 00f9 00f6 00061583 GPR08: c0398070 8072f930 c0398070 c03978c0 004e 415ec006 1ffb3300 GPR16: 1ffa6e70 1ffad780 c036b1a0 ff1ad8aa c038bf48 GPR24: 00029000 001c 262a1965 c036de50 c039269c fffef700 fffef700 2625a000 NIP [c002eac4] get_next_timer_interrupt+0x30/0x260 LR [c004811c] tick_nohz_stop_sched_tick+0x110/0x404 Call Trace: [c038bf10] [c003df74] ktime_get+0x1c/0x44 (unreliable) [c038bf40] [c004811c] tick_nohz_stop_sched_tick+0x110/0x404 [c038bf90] [c0006f00] cpu_idle+0x50/0xd8 [c038bfb0] [c000197c] rest_init+0x5c/0x70 [c038bfc0] [c033f844] start_kernel+0x224/0x2a0 [c038bff0] [c200] skpinv+0x190/0x1cc Instruction dump: 9421ffd0 7c0802a6 bfa10024 90010034 3d20c037 8169d1b8 7c7e1b78 808b0004 5485063e 7ca62b78 54c91838 7d295a14 <81090008> 39290008 4814 80080014 ---[ end trace 13e96cd635551e49 ]--- Kernel panic - not syncing: Attempted to kill the idle task! Rebooting in 1 seconds..aym...@vbox:~$ ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Help with an odd problem sharing memory on the ppc460ex
I've got custom boards that have been running for a while on rev A 460ex parts but when the rev B parts became available some problems surfaced. We are trying to work around the issues in software. To make this simple, I've got 2 460exs connected together via PCI and PCIe so i can switch the transport. For now, I am using PCI. I've setup the PIMs and POMs to map one CPUs DRAM across the bus to the other CPU. So they have a sort of shared memory scheme. This works fine in many cases. I think I have a caching problem though even though I think the cache is disabled via u-boot. What happens is that CPU0 will write into CPU1's memory. CPU1 will still see a stale value though ... I can't figure out how this happens. If CPU0 comes back and reads from CPU1's memory though, it gets the correct value. Now if I reduce the frequency of the reads/writes to something low (maybe 1 or 2 per second), then very often CPU1 will see the change. I continue to think it is a cache issue, but I am not sure. Also not sure how to work around that. Is that something I need to do via the TLBs? Is the cache really disabled or does Linux mess with the TLB/cache settings? When I boot Linux I reserve the upper 16Mb of memory (from 112-128Mb space) and then mmap that entire region into CPU1's space. Not sure if when it is reserved if it is treated differently by Linux or not. Any help is greatly appreciated. Thanks Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: Problems mapping OCM memory from 460EX to user space
On 7/7/2010 2:48 PM, ay...@austin.rr.com wrote: rc = io_remap_pfn_range(vma, PLB_OCM_BASE_ADDR>>PAGE_SHIFT, vma->vm_start, len, vma->vm_page_prot); I am fairly certain the physical address is correct, I've verified that the TLB entries in u-boot look ok. Any tips on how to make this work? I found / fixed the issue. The first problem is that start and offset are swapped in the io_remap_pfn_range invocation. The 2nd was that there were some TLB changes that were needed to map the OCM. ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: How to set GPIO state in the dts or in platform
On 1/5/2010 10:38 PM, Bill Gatliff wrote:
Ayman El-Khashab wrote:
I've got a custom board akin to the walnut. I've successfully got
u-boot,
the kernel, and a working filesystem. The only thing I lack is a
clear idea
of how to set a GPIO.
Isn't that the "flags" parameter, e.g.:
gpios =
?
If you are passing a 0 for the flags, try a 1. I know of at least one
driver that interprets that to mean a polarity inversion. Maybe you'll
get lucky. :)
Possibly, I saw that in the documentation directory, but didn't quite
follow the explanation that
was given. Below is what I was using as a reference, but is it as
simple as that? Do the flags encode
just the state or also, the open drain, hi-z and everything else? In my
case, I want the GPIO 3 actively
driven high.
Example of the node using GPIOs:
20
21 node {
22 gpios = <&qe_pio_e 18 0>;
23 };
24
25 In this example gpio-specifier is "18 0" and encodes GPIO pin number,
26 and empty GPIO flags as accepted by the "qe_pio_e" gpio-controller.
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
How to set GPIO state in the dts or in platform
I've got a custom board akin to the walnut. I've successfully got u-boot, the kernel, and a working filesystem. The only thing I lack is a clear idea of how to set a GPIO. I have the PHY reset connected to GPIO 3. I've got it setup in u-boot correctly, but by the time linux boots, it has touched the GPIOs and now the PHY is held in reset. I tried to follow how to adjust my device tree, but I haven't been able to get it to work yet. My kernel (circa mid 2008) doesn't have the "keep" property of the GPIO node, so I can't use that. Based on what I've read it seems that I might need to add a node to the dts and then a custom piece of code in the platform directory for my board. I am just not sure how to proceed there. thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
help with unhandled IRQ error with mpt2sas driver and powerpc 460EX
Hello, I am using the mpt2sas driver on the canyonlands / 460EX board.
I've already found and fixed one problem, but can't get past the
unhandled IRQ.
The first problem I noticed is that the physical address is read into a
32 bit variable, but the 460ex has a 36 bit bus so the ioremap would
always fail. I've change the defn of chip_phys in mpt2sas_base.h to u64
and that cleared up that issue. As soon as the unmask_interrupts
method is called (or not long after), I get an interrupt -- presumably
from the sas controller. If I comment out the unmask, the interrupt
never occurs. If I unmask them, I get the interrupt. I've traced the
code through the interrupt handler all the way to ~ line 757.
rpf = &ioc->reply_post_free[ioc->reply_post_host_index];
I've verified that at the end of this, IRQ_NONE is returned. At this
point the kernel prints the following -- the last statements lead me to
think that the sas controller expected something but never got it. I am
unsure how to proceed at this point. I am using a denx kernel head
pulled from git today since there were some changes to thsi driver for
endian issues.
irq 18: nobody cared (try booting with the "irqpoll" option)
Call Trace:
[c0367df0] [c0005eac] show_stack+0x44/0x16c (unreliable)
[c0367e30] [c004eedc] __report_bad_irq+0x34/0xb8
[c0367e50] [c004f118] note_interrupt+0x1b8/0x224
[c0367e80] [c004ff50] handle_level_irq+0xa0/0x11c
[c0367e90] [c0018ba4] uic_irq_cascade+0xf8/0x12c
[c0367eb0] [c00041d0] do_IRQ+0x98/0xb4
[c0367ed0] [c000df40] ret_from_except+0x0/0x18
[c0367f90] [c0006ed8] cpu_idle+0x50/0xd8
[c0367fb0] [c000197c] rest_init+0x5c/0x70
[c0367fc0] [c0320848] start_kernel+0x224/0x2a0
[c0367ff0] [c200] skpinv+0x190/0x1cc
handlers:
[] (_base_interrupt+0x0/0x8f8)
Disabling IRQ #18
mpt2sas0: _base_event_notification: timeout
mf:
0700 0f2f3fff fffc
mpt2sas0: sending diag reset !!
mpt2sas0: diag reset: SUCCESS
mpt2sas0: failure at
drivers/scsi/mpt2sas/mpt2sas_scsih.c:5989/_scsih_probe()!
Thanks
Ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Trouble with canyonlands and PCI-E LSI SAS Controller
Hello, I've got some trouble getting an LSI SAS controller (LSISAS200) to function. It is at least enumerated correctly in a PC running ubuntu, so it seems that it must be something particular to the PCI implementation on the powerpc or the usage of u-boot. If the board is initialized by u-boot, and the BAR0 is written, then the board does not seem to respond to cfgrds when linux probes. And the kernel crashes when the scsi low level are loaded. If u-boot is inhibited and therefore linux sees the card in its reset state then it will boot ok, but an lspci will show the card as disabled. The last issue, is that I am not quite sure exactly which kernel drivers are needed for this device. Here is the output from DEBUG=1 in the pci_probe. The first is without the fusion driver, and the latter is with the fusion driver. Thanks Ayman # dmesg Bus: primary=80, secondary=81, subordinate=bf, sec-latency=0 Using PowerPC 44x Platform machine description Linux version 2.6.31 (b...@lablinux) (gcc version 4.2.2) #61 Tue Oct 6 08:25:39 CDT 2009 Found initrd at 0xdfb09000:0xdfe4cc6eidge: 8010-801f Found legacy serial port 0 for /plb/opb/ser...@ef600300 mem=4ef600300, taddr=4ef600300, irq=0, clk=6451612, speed=0 Found legacy serial port 1 for /plb/opb/ser...@ef600400: Mask- 64bit+ Queue=0/2 Enable- mem=4ef600400, taddr=4ef600400, irq=0, clk=6451612, speed=0 Found legacy serial port 2 for /plb/opb/ser...@ef600500 mem=4ef600500, taddr=4ef600500, irq=0, clk=6451612, speed=0 Found legacy serial port 3 for /plb/opb/ser...@ef600600 mem=4ef600600, taddr=4ef600600, irq=0, clk=6451612, speed=0 Top of RAM: 0x2000, Total RAM: 0x2000 Memory hole size: 0MBevsel, IRQ 18 Zone PFN ranges:s at 1000 [disabled] [size=256] DMA 0x -> 0x0002, non-prefetchable) [disabled] [size=16K] Normal 0x0002 -> 0x0002, non-prefetchable) [disabled] [size=256K] HighMem 0x0002 -> 0x0002e8010 [disabled] [size=512K] Movable zone start PFN for each nodeagement version 3 early_node_map[1] active PFN rangesEndpoint IRQ 0 0: 0x -> 0x0002l Product Data On node 0 totalpages: 131072essage Signalled Interrupts: Mask- 64bit+ Queue=0/0 Enable- free_area_init_node: node 0, pgdat c0335654, node_mem_map c0363000 DMA zone: 1024 pages used for memmaprror Reporting DMA zone: 0 pages reservedPower Budgeting DMA zone: 130048 pages, LIFO batch:31 MMU: Allocated 1088 bytes of context maps for 255 contexts Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Kernel command line: root=/dev/ram rw mem=128M mem=512M ip=169.254.0.181:169.254.0.100:169.254.0 PID hash table entries: 2048 (order: 11, 8192 bytes) Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) High memory: 0k Memory: 512512k/524288k available (3168k kernel code, 11356k reserved, 128k data, 132k bss, 160) Kernel virtual memory layout: * 0xfffef000..0xf000 : fixmap * 0xffc0..0xffe0 : highmem PTEs * 0xffa0..0xffc0 : consistent mem * 0xffa0..0xffa0 : early ioremap * 0xe100..0xffa0 : vmalloc & ioremap SLUB: Genslabs=11, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 NR_IRQS:512 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 irq: irq 30 on host /interrupt-controller0 mapped to virtual irq 30 UIC2 (32 IRQ sources) at DCR 0xe0 irq: irq 10 on host /interrupt-controller0 mapped to virtual irq 16 UIC3 (32 IRQ sources) at DCR 0xf0 irq: irq 16 on host /interrupt-controller0 mapped to virtual irq 17 time_init: decrementer frequency = 800.10 MHz time_init: processor frequency = 800.10 MHz clocksource: timebase mult[50] shift[22] registered clockevent: decrementer mult[ccf7] shift[32] cpu[0] Mount-cache hash table entries: 512 NET: Registered protocol family 16 irq: irq 23 on host /interrupt-controller2 mapped to virtual irq 23 256k L2-cache enabled PCIE0: Checking link... PCIE0: No device detected. PCI host bridge /plb/pc...@d (primary) ranges: MEM 0x000e..0x000e7fff -> 0x8000 IO 0x000f8000..0x000f8000 -> 0x 4xx PCI DMA offset set to 0x PCIE0: successfully set as root-complex PCIE1: Checking link... PCIE1: Device detected, waiting for link... PCIE1: link is up ! PCI host bridge /plb/pc...@d2000 (primary) ranges: MEM 0x000e8000..0x000e -> 0x8000 IO 0x000f8001..0x000f8001 -> 0x 4xx PCI DMA offset set to 0x PCIE1: successfully set as root-complex PCI host bridge /plb/p...@c0ec0 (primary) ranges: MEM 0x000d8000..0x000d -> 0x8000 IO 0x000c0800..0x000c0800 -> 0x 4xx PCI DMA offset set to 0x PCI: Probing PCI hardware PCI: Scanning bus :40 pci :40:00.0: found [aaa0:bed0] class 00
Trying to understand ppc4xx_configure_pciex_PIMs mapping to physical address 0
I am using the ppc460ex as an endpoint and I see that the mentioned function setups a 32Mb window at address 0 in sdram. what i want is some memory that the host can read/write to. But the 32Mb that are already mapped at the bottom of ram, so it would not be good do writes to those addresses. Am I missing something? My driver right now is not touching the PIM registers. So is the right way to do this to change the PIM registers after I get a buffer in the kernel -- or something else? How would I make sure that it was aligned on a 32Mb boundry per the 44x requirements? thanks ayman ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
RE: Help enabling PCI endpoint on 460EX, host sees disabled
> -Original Message- > From: Ira Snyder [mailto:[EMAIL PROTECTED] > Sent: Wednesday, December 10, 2008 6:18 PM > To: Ayman El-Khashab > Cc: linuxppc-dev@ozlabs.org > Subject: Re: Help enabling PCI endpoint on 460EX, host sees disabled > > > I can't help you with that exact board, but I've written an "ethernet > over PCI" driver for Linux and U-Boot. The Linux driver has > been posted > here on linuxppc-dev and on the lkml. Search for "net: add PCINet > driver" and you'll find it. Hello Ira, I am familiar with it, I looked through all of your posts and the comments from RFCs 1-4. That is what I was modeling mine after when I ran across the "disabled" problem. One thing I did not understand was the device tree entries you made -- though that is unrelated to my problem. > Definitely get PCI/PCI-E working and stable before you try to start > communicating between the devices, though. Thats my goal, there are a couple of things I am not sure about so I'm sure I'll post again. ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Help enabling PCI endpoint on 460EX, host sees disabled
My system consists of a pair of 460EXs attached by way of both PCI-E and PCI. Ultimately my goal is to communicate between them via pci-e (is there anything out there that does this already?). For a small step, I am trying to get the PCI to work. I already have the PCI boot ability of the 460EXs working so the "adapter" processor is booting from the masters main memory. Right now I am trying to just exchange data via the regular PCI bus. I've looked through the kernel code and see where it is using the "dma-ranges" from the device tree to setup the PIM registers. So I am good there. When I look at the PCI bus from the host though, I see the following output from lspci. 0001:00:02.0 Class 0680: Unknown device 10e8:a003 Subsystem: Unknown device 1014:cafe Flags: bus master, 66MHz, medium devsel, latency 128, IRQ 22 Memory at d8000 (64-bit, prefetchable) [disabled] [size=8000] Capabilities: [c0] Message Signalled Interrupts: Mask- 64bit+ Queue=0/2 Enable- Capabilities: [d0] Power Management version 2 Capabilities: [dc] PCI-X non-bridge device Capabilities: [48] Vital Product Data I am assuming that the disabled is probably because the pci command register on the adapter is not correct. However I am not entirely sure. And I assume that the disabled means I won't be able to write to it. So what do I need to do to get this enabled? Thanks ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
RE: [RFC/PATCH] pci: Workaround invalid P2P bridge bus numbers
Benjamin Herrenschmidt wrote: > > Ayman, can you double check that this variant of the patch still > fixes your problem ? Thanks ! > I've tried it out and it is working correctly with my devices. The subordinate bus numbers on all the ports are correct. Best Regards, Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
RE: Problems with PCI-E devices not being detected with switch
Benjamin Herrenschmidt wrote:
>>>
>>> + /* Check if setup is sensible at all */
>>> + if ((buses & 0xff) != bus->number ||
>>> + ((buses >> 8) & 0xff) != <= bus->number) {
>>
>> Note that I removed the <= from the above line -- I did not think it
>> was correct. Please let me know if that was wrong.
>
> My logic is that the current setup is incorrect if the primary bus of
> the bridge doesn't match the parent bus number, or if the secondary
> bus number of the bridge is not strictly superior to the parent bus
> number.
>
> What sounds incorrect ?
>
The part that didn't look correct is this line (note the operators)
((buses >> 8) & 0xff) != <= bus->number) {
Operators ^^ ^^
>From reading through the code and your textual description of what was
supposed to be happening, I went ahead and changed it to ...
((buses >> 8) & 0xff) != bus->number) {
And this is the code that generated the results from my previous
message.
Hope that made sense ...
Regards,
Ayman
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
RE: Problems with PCI-E devices not being detected with switch
Thank you Ben, I've had success with the patch, details below ...
Benjamin Herrenschmidt wrote:
> Can you try this (untested) patch and send me the resulting dmesg log
> (along with whether it helps).
>
> Index: linux-work/drivers/pci/probe.c
> ===
> --- linux-work.orig/drivers/pci/probe.c 2008-10-18 08:10:25.0
> +1100 +++ linux-work/drivers/pci/probe.c 2008-10-18
> 08:14:14.0 +1100 @@ -536,19 +536,27 @@
> int is_cardbus = (dev->hdr_type == PCI_HEADER_TYPE_CARDBUS);
> u32 buses, i, j = 0;
> u16 bctl;
> + int broken = 0;
>
> pci_read_config_dword(dev, PCI_PRIMARY_BUS, &buses);
>
> dev_dbg(&dev->dev, "scanning behind bridge, config %06x, pass %d\n",
> buses & 0xff, pass);
>
> + /* Check if setup is sensible at all */
> + if ((buses & 0xff) != bus->number ||
> + ((buses >> 8) & 0xff) != <= bus->number) {
Note that I removed the <= from the above line -- I did not think it was
correct. Please let me know if that was wrong.
Attached below is the dmesg output, in this case the SIL on the downstream
switch port was detected correctly. Advise if there is something else that
needs to be tried.
Thanks
Ayman
Kernel command line: ramdisk_size=65536 root=/dev/ram rw ip=169.254.0.199:169.20
UIC0 (32 IRQ sources) at DCR 0xc0
UIC1 (32 IRQ sources) at DCR 0xd0
UIC2 (32 IRQ sources) at DCR 0xe0
UIC3 (32 IRQ sources) at DCR 0xf0
PID hash table entries: 2048 (order: 11, 8192 bytes)
time_init: decrementer frequency = 800.10 MHz
time_init: processor frequency = 800.10 MHz
clocksource: timebase mult[50] shift[22] registered
clockevent: decrementer mult[] shift[16] cpu[0]
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
High memory: 0k
Memory: 512640k/524288k available (3244k kernel code, 11336k reserved, 124k dat)
SLUB: Genslabs=10, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Calibrating delay loop... 1597.44 BogoMIPS (lpj=3194880)
Mount-cache hash table entries: 512
net_namespace: 288 bytes
NET: Registered protocol family 16
256k L2-cache enabled
PCIE0: Checking link...
PCIE0: No device detected.
PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges:
MEM 0x000e..0x000e7fff -> 0x8000
IO 0x000f8000..0x000f8000 -> 0x
4xx PCI DMA offset set to 0x
PCIE0: successfully set as endpoint
PCIE1: Checking link...
PCIE1: Device detected, waiting for link...
PCIE1: link is up !
PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges:
MEM 0x000e8000..0x000e -> 0x8000
IO 0x000f8001..0x000f8001 -> 0x
4xx PCI DMA offset set to 0x
PCIE1: successfully set as root-complex
PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges:
MEM 0x000d8000..0x000d -> 0x8000
IO 0x000c0800..0x000c0800 -> 0x
4xx PCI DMA offset set to 0x
PCI: Probing PCI hardware
PCI: Scanning bus :40
pci :40:00.0: found [eee0/fed0] class 000b20 header type 00
PCI: :40:00.0 reg 10 64bit mmio: [0, 1ff]
PCI: Hiding 4xx host bridge resources :40:00.0
PCI: Fixups for bus :40
PCI: Bus scan for :40 returning with max=40
PCI: Scanning bus 0001:80
pci 0001:80:00.0: found [aaa1/bed1] class 000604 header type 01
PCI: 0001:80:00.0 reg 10 32bit mmio: [0, 7fff]
PCI: Hiding 4xx host bridge resources 0001:80:00.0
PCI: Fixups for bus 0001:80
pci 0001:80:00.0: scanning behind bridge, config bf8180, pass 0
pci 0001:80:00.0: bus configuration doesn't match, reconfiguring
pci 0001:80:00.0: scanning behind bridge, config 00, pass 1
pci 0001:80:00.0: bus configuration doesn't match, reconfiguring
PCI: Scanning bus 0001:81
pci 0001:81:00.0: found [10b5/8509] class 000604 header type 01
PCI: 0001:81:00.0 reg 10 32bit mmio: [b800, b801]
pci 0001:81:00.0: PME# supported from D0 D3hot D3cold
pci 0001:81:00.0: PME# disabled
PCI: Fixups for bus 0001:81
PCI: bridge 0001:80:00.0 io port: [0, fff]
PCI: bridge 0001:80:00.0 32bit mmio: [b800, b81f]
pci 0001:81:00.0: scanning behind bridge, config 060201, pass 0
pci 0001:81:00.0: bus configuration doesn't match, reconfiguring
pci 0001:81:00.0: scanning behind bridge, config 00, pass 1
pci 0001:81:00.0: bus configuration doesn't match, reconfiguring
PCI: Scanning bus 0001:82
pci 0001:82:01.0: found [10b5/8509] class 000604 header type 01
pci 0001:82:01.0: PME# supported from D0 D3hot D3cold
pci 0001:82:01.0: PME# disabled
pci 0001:82:02.0: found [10b5/8509] class 000604 header type 01
pci 0001:82:02.0: PME# supported from D0 D3hot D3cold
pci 0001:82:02.0: PME# disabled
pci 0001:82:03.0: found [10b5/8509] class 000604 header type 01
pci 0001:82:03.0: PME# supported from D0 D3hot D3cold
pci 0001:82:03.0: PME# disabled
RE: Problems with PCI-E devices not being detected with switch
Benjamin Herrenschmidt wrote: > On Thu, 2008-10-16 at 10:01 -0500, Ayman El-Khashab wrote: >> Benjamin Herrenschmidt wrote: >>> On Wed, 2008-10-15 at 10:47 -0500, Ayman El-Khashab wrote: >>> >>> Note for people on CC: This is a problem on 460EX on a canyonland >>> using the 4x port. >>> > > Ok, can you send me a full dmesg log with "debug" on the kernel > command line after adding a #define DEBUG 1 to the top of > drivers/pci/probe.c please ? (before the batch of #include). > Yes, it is below. I saw the #define DEBUG 1 turned on these messages below, but I am not sure how to verify that I correctly added the "debug" to the kernel arguments. (In u-boot I added them to the end of the ramargs variable) thanks, - ame # dmesg Linux version 2.6.27-01160-gc08fd34-dirty ([EMAIL PROTECTED]) (gcc version 4.2.2) #11 Fri Oct 17 09:32:16 CDT 2008 Found initrd at 0xc6cd5000:0xc6fffbc7 Found legacy serial port 0 for /plb/opb/[EMAIL PROTECTED] mem=4ef600300, taddr=4ef600300, irq=0, clk=7407407, speed=0 Found legacy serial port 1 for /plb/opb/[EMAIL PROTECTED] mem=4ef600400, taddr=4ef600400, irq=0, clk=7407407, speed=0 Found legacy serial port 2 for /plb/opb/[EMAIL PROTECTED] mem=4ef600500, taddr=4ef600500, irq=0, clk=7407407, speed=0 Found legacy serial port 3 for /plb/opb/[EMAIL PROTECTED] mem=4ef600600, taddr=4ef600600, irq=0, clk=7407407, speed=0 Top of RAM: 0x2000, Total RAM: 0x2000 Memory hole size: 0MB Zone PFN ranges: DMA 0x -> 0x0002 Normal 0x0002 -> 0x0002 HighMem 0x0002 -> 0x0002 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x -> 0x0002 On node 0 totalpages: 131072 free_area_init_node: node 0, pgdat c0344a78, node_mem_map c0372000 DMA zone: 130048 pages, LIFO batch:31 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 130048 Kernel command line: ramdisk_size=65536 root=/dev/ram rw debug ip=169.254.0.102: 169.254.0.100:169.254.0.100:255.255.255.0:tanosx:eth0:off panic=1 console=ttyS0, 115200 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 UIC2 (32 IRQ sources) at DCR 0xe0 UIC3 (32 IRQ sources) at DCR 0xf0 PID hash table entries: 2048 (order: 11, 8192 bytes) time_init: decrementer frequency = 800.10 MHz time_init: processor frequency = 800.10 MHz clocksource: timebase mult[50] shift[22] registered clockevent: decrementer mult[] shift[16] cpu[0] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) High memory: 0k Memory: 512640k/524288k available (3228k kernel code, 11320k reserved, 124k data , 138k bss, 160k init) SLUB: Genslabs=10, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Calibrating delay loop... 1597.44 BogoMIPS (lpj=3194880) Mount-cache hash table entries: 512 net_namespace: 288 bytes NET: Registered protocol family 16 256k L2-cache enabled PCIE0: Checking link... PCIE0: No device detected. PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges: MEM 0x000e..0x000e7fff -> 0x8000 IO 0x000f8000..0x000f8000 -> 0x 4xx PCI DMA offset set to 0x PCIE0: successfully set as endpoint PCIE1: Checking link... PCIE1: Device detected, waiting for link... PCIE1: link is up ! PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges: MEM 0x000e8000..0x000e -> 0x8000 IO 0x000f8001..0x000f8001 -> 0x 4xx PCI DMA offset set to 0x PCIE1: successfully set as root-complex PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges: MEM 0x000d8000..0x000d -> 0x8000 IO 0x000c0800..0x000c0800 -> 0x 4xx PCI DMA offset set to 0x PCI: Probing PCI hardware PCI: Scanning bus :40 pci :40:00.0: found [eee0/fed0] class 000b20 header type 00 PCI: :40:00.0 reg 10 64bit mmio: [0, 1ff] PCI: Hiding 4xx host bridge resources :40:00.0 PCI: Fixups for bus :40 PCI: Bus scan for :40 returning with max=40 PCI: Scanning bus 0001:80 pci 0001:80:00.0: found [aaa1/bed1] class 000604 header type 01 PCI: 0001:80:00.0 reg 10 32bit mmio: [0, 7fff] PCI: Hiding 4xx host bridge resources 0001:80:00.0 PCI: Fixups for bus 0001:80 pci 0001:80:00.0: scanning behind bridge, config bf8180, pass 0 PCI: Scanning bus 0001:81 pci 0001:81:00.0: found [10b5/8509] class 000604 header type 01 PCI: 0001:81:00.0 reg 10 32bit mmio: [b800, b801] pci 0001:81:00.0: PME# supported from D0 D3hot D3cold pci 0001:81:00.0: PME# disabled PCI: Fixups for bus 0001:81 PCI: bridge 0001:80:00.0 io port: [0, fff] PCI: bridge 0001:80:00.0 32bit mmio: [b800, b81f] pci 0001:81:00.0: scanning behind bridge, config 060201, pass 0 PCI: Scanning bus 0001:02 pci 00
RE: Problems with PCI-E devices not being detected with switch
Benjamin Herrenschmidt wrote: > On Wed, 2008-10-15 at 10:47 -0500, Ayman El-Khashab wrote: > > Note for people on CC: This is a problem on 460EX on a canyonland > using the 4x port. > >> The problem occurs when Linux boots. It sees the switch (and looking >> in /sys/bus/... confirms it), but nothing on the downstream sides of >> the switch (secondary busses) is visible. There were no boot >> messages to indicate it had seen the Sil 3531 and it doesn't >> function. We've also tried other PCI-E devices (NI GPIB) on the >> downstream side and they are also not detected, so it seems to be >> something in Linux, my configuration, etc. I've included the boot >> messages below from u-boot and the kernel. It is more than just the >> pci boot messages, but I was not sure if something else in the log >> with provide some insight. > > The messages below look really fishy indeed: > >> pci 0001:02:00.0: unknown header type 03, ignoring device pci >> 0001:02:01.0: unknown header type 41, ignoring device pci >> 0001:02:02.0: unknown header type 03, ignoring device pci >> 0001:02:03.0: unknown header type 41, ignoring device pci >> 0001:02:04.0: ignoring class 1d09 (doesn't match header type 02) pci >> 0001:02:05.0: unknown header type 41, ignoring device pci >> 0001:02:06.0: ignoring class 1d09 (doesn't match header type 02) pci >> 0001:02:07.0: unknown header type 41, ignoring device pci >> 0001:02:08.0: unknown header type 03, ignoring device pci >> 0001:02:09.0: unknown header type 41, ignoring device pci >> 0001:02:0a.0: ignoring class 7d09 (doesn't match header type 02) pci >> 0001:02:0b.0: unknown header type 41, ignoring device pci >> 0001:02:0c.0: ignoring class 1d01 (doesn't match header type 02) pci >> 0001:02:0d.0: unknown header type 41, ignoring device pci >> 0001:02:0e.0: ignoring class 7d09 (doesn't match header type 02) pci >> 0001:02:0f.0: unknown header type 41, ignoring device pci >> 0001:02:10.0: unknown header type 03, ignoring device pci >> 0001:02:11.0: unknown header type 41, ignoring device pci >> 0001:02:12.0: unknown header type 03, ignoring device pci >> 0001:02:13.0: unknown header type 41, ignoring device pci >> 0001:02:14.0: unknown header type 03, ignoring device pci >> 0001:02:15.0: unknown header type 41, ignoring device pci >> 0001:02:16.0: ignoring class 3d09 (doesn't match header type 02) pci >> 0001:02:17.0: unknown header type 41, ignoring device pci >> 0001:02:18.0: unknown header type 03, ignoring device pci >> 0001:02:19.0: unknown header type 41, ignoring device pci >> 0001:02:1a.0: unknown header type 03, ignoring device pci >> 0001:02:1b.0: unknown header type 41, ignoring device pci >> 0001:02:1c.0: ignoring class 5d01 (doesn't match header type 02) pci >> 0001:02:1d.0: unknown header type 41, ignoring device pci >> 0001:02:1e.0: unknown header type 03, ignoring device pci >> 0001:02:1f.0: unknown header type 41, ignoring device > > > Ayman, can you try adding a long delay (such as msleep(5000), ie 5s) > at the beginning of pcibios_init() in arch/powerpc/kernel/pci_32.c ? > This will add 5s delay between the init/reset of the port and the > probing by linux. Do that help ? > Thanks for your help. I did try a 5s delay and other than the boot taking an extra couple of seconds, there was no change. The messages above "unknown header" still showed up. Downstream devices were not found. BTW, this is with both the Denx kernel tree and the official 2.6.27 kernel. Both have the same results. One added note, just to rule out bad electronics, I have confirmed that the switch+sata work correctly together if placed into a plain x86 pc with 2.6.24. So the electronics are fine. Ayman ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Problems with PCI-E devices not being detected with switch
I am using the stock canyonlands 460EX development board. I have the 4L (port 1) configured as a root-complex. I also have a SIL pci-e sata card. When I plug the sata card directly into the canyonlands, everything works as expected. The sil is detected in Linux and I can read/write the disk. The sil is a 3531. The next step is to place this on the other side of a PCI-E switch. The particular switch is a PLX 8509. I connect this to the 4L port and then connect the sil to a 1L port on the 8509. When u-boot comes up, it detects all of the switch ports and the 3531 correctly. The problem occurs when Linux boots. It sees the switch (and looking in /sys/bus/... confirms it), but nothing on the downstream sides of the switch (secondary busses) is visible. There were no boot messages to indicate it had seen the Sil 3531 and it doesn't function. We've also tried other PCI-E devices (NI GPIB) on the downstream side and they are also not detected, so it seems to be something in Linux, my configuration, etc. I've included the boot messages below from u-boot and the kernel. It is more than just the pci boot messages, but I was not sure if something else in the log with provide some insight. Thanks Ayman -- AND: 128 MiB PCI: Bus Dev VenId DevId Class Int PCIE0: link is not up. PCIE0: initialization as root-complex failed PCIE1: successfully set as root-complex 03 01 10b5 8509 0604 00 03 02 10b5 8509 0604 00 06 00 1095 3531 0180 00 03 03 10b5 8509 0604 00 03 04 10b5 8509 0604 00 02 00 10b5 8509 0604 00 Net: ppc_4xx_eth0, ppc_4xx_eth1 Type run flash_nfs to mount root filesystem over NFS Hit any key to stop autoboot: 0 Waiting for PHY auto negotiation to complete.. done ENET Speed is 100 Mbps - FULL duplex connection (EMAC0) BOOTP broadcast 1 BOOTP broadcast 2 BOOTP broadcast 3 *** Unhandled DHCP Option in OFFER/ACK: 7 *** Unhandled DHCP Option in OFFER/ACK: 44 *** Unhandled DHCP Option in OFFER/ACK: 7 *** Unhandled DHCP Option in OFFER/ACK: 44 DHCP client bound to address 169.254.0.102 ## Booting kernel from Legacy Image at fc00 ... Image Name: Linux-2.6.27-rc6-01149-g59c72dc- Created: 2008-09-18 22:17:29 UTC Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size:1616554 Bytes = 1.5 MB Load Address: Entry Point: Verifying Checksum ... OK Uncompressing Kernel Image ... OK ## Flattened Device Tree blob at fc1e Booting using the fdt blob at 0xfc1e ## Loading init Ramdisk from Legacy Image at fc20 ... Image Name: Tanisys Ramdisk Image Created: 2008-09-18 20:29:45 UTC Image Type: PowerPC Linux RAMDisk Image (gzip compressed) Data Size:3320775 Bytes = 3.2 MB Load Address: Entry Point: Verifying Checksum ... OK Loading Device Tree to 007fc000, end 007ff0c0 ... OK Loading Ramdisk to 06cd5000, end 06fffbc7 ... OK Using Canyonlands machine description Linux version 2.6.27-rc6-01149-g59c72dc-dirty ([EMAIL PROTECTED]) (gcc version 4.2.2) #9 Thu Sep 18 17:17:19 CDT 2008 Found initrd at 0xc6cd5000:0xc6fffbc7 Zone PFN ranges: DMA 0x -> 0x7000 Normal 0x7000 -> 0x7000 HighMem 0x7000 -> 0x7000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x -> 0x7000 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 28448 Kernel command line: ramdisk_size=65536 root=/dev/ram rw mem=112M ip=169.254.0.1 02:169.254.0.100:169.254.0.100:255.255.255.0:tanosx:eth0:off panic=1 console=tty S0,115200 UIC0 (32 IRQ sources) at DCR 0xc0 UIC1 (32 IRQ sources) at DCR 0xd0 UIC2 (32 IRQ sources) at DCR 0xe0 UIC3 (32 IRQ sources) at DCR 0xf0 PID hash table entries: 512 (order: 9, 2048 bytes) clocksource: timebase mult[50] shift[22] registered Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) Memory: 106764k/114688k available (3188k kernel code, 7784k reserved, 124k data, 138k bss, 160k init) SLUB: Genslabs=10, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Calibrating delay loop... 1597.44 BogoMIPS (lpj=3194880) Mount-cache hash table entries: 512 net_namespace: 288 bytes NET: Registered protocol family 16 256k L2-cache enabled PCIE0: Checking link... PCIE0: No device detected. PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges: MEM 0x000e..0x000e7fff -> 0x8000 IO 0x000f8000..0x000f8000 -> 0x 4xx PCI DMA offset set to 0x PCIE0: successfully set as endpoint PCIE1: Checking link... PCIE1: Device detected, waiting for link... PCIE1: link is up ! PCI host bridge /plb/[EMAIL PROTECTED] (primary) ranges: MEM 0x000e8000..0x000e -> 0x8000 IO 0x000f8001..0x000f8001 -> 0x 4xx PCI DMA offset set to
