Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip
On Jun 24, 2008, at 10:51 AM, Vince Asbridge wrote: All, I'm new to this mailing list, but have not had any luck finding information on this issue. Please be kind if I break the forum rules on my first post. We recently tried to upgrade our Freescale CDS 8548 look-alike module (code name ATCA1000) from the 2.6.11 based BSP to the 2.6.23 based BSP. The upgrade went fairly smoothly, until we tried using SOME pci-e devices (some work fine, some don't show up to lspci). LSI pci-e controllers no longer show up at all! We see the ixgbe (intel 10G), SiliconImage SATA controller but do not see LSI devices (Specifically 1068 SAS, FC949-E fibrechannel). We're guessing it's a resource issue behind the bridge, because the LSI devices try to allocate 1 - 3M behind the bridge, but we can't find the bug, or where we would debug such an issue. The devices seem to "train" correctly, because we have an LED on the pci-e switch (PLX 8 port pci-e switch), and it's ON indicating pci-e link between the bridge and the 1068 device). We're totally at a loss as to why this always worked on the 2.6.11 kernel but doesn’t work on 2.6.23. Using lspci, the LSI adapters do not show up in the list at all, as though they are not plugged into the system. Is there something that needs to be done with respect to PCI-E devices that is new in the 2.6.23 based BSP that did not need to be done in the 2.6.11 based kit? For example, are pci resources allocated by a different piece of code, that may have some issue allocating resources for the LSI adapters? Can you tree 2.6.25? There are some fixes in that kernel related to PCIe support: [POWERPC] FSL: Rework PCI/PCIe support for 85xx/86xx However, its odd that lspci doesn't even show the device. Are you using u-boot? if so what version? and does u-boot see the LSI? - k ___ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded
Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip
I am also having problems with PCI-E device LSI 1064E on 2.6.24-rc6
kernel. This is running on 8641D processor.
I can see the device, access the config space, but seems access from
device to memory is not working. Same device (same exact hardware) works
fine for 2.6.15 kernel.
- Siva
-Original Message-
Date: Wed, 25 Jun 2008 10:30:35 -0500
From: Kumar Gala <[EMAIL PROTECTED]>
Subject: Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI
1068 SAS chip
To: Vince Asbridge <[EMAIL PROTECTED]>
Cc: linuxppc-embedded@ozlabs.org
Message-ID: <[EMAIL PROTECTED]>
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed;
delsp=yes
On Jun 24, 2008, at 10:51 AM, Vince Asbridge wrote:
> All,
>
> I'm new to this mailing list, but have not had any luck finding
> information on this issue.
>
> Please be kind if I break the forum rules on my first post.
>
> We recently tried to upgrade our Freescale CDS 8548 look-alike
> module (code name ATCA1000) from the 2.6.11 based BSP to the 2.6.23
> based BSP.
>
> The upgrade went fairly smoothly, until we tried using SOME pci-e
> devices (some work fine, some don't show up to lspci).
>
> LSI pci-e controllers no longer show up at all!
>
> We see the ixgbe (intel 10G), SiliconImage SATA controller but do
> not see LSI devices (Specifically 1068 SAS, FC949-E fibrechannel).
>
> We're guessing it's a resource issue behind the bridge, because the
> LSI devices try to allocate 1 - 3M behind the bridge, but we can't
> find the bug, or where we would debug such an issue.
>
> The devices seem to "train" correctly, because we have an LED on the
> pci-e switch (PLX 8 port pci-e switch), and it's ON indicating pci-e
> link between the bridge and the 1068 device).
>
> We're totally at a loss as to why this always worked on the 2.6.11
> kernel but doesn?t work on 2.6.23.
>
> Using lspci, the LSI adapters do not show up in the list at all, as
> though they are not plugged into the system.
>
> Is there something that needs to be done with respect to PCI-E
> devices that is new in the 2.6.23 based BSP that did not need to be
> done in the 2.6.11 based kit? For example, are pci resources
> allocated by a different piece of code, that may have some issue
> allocating resources for the LSI adapters?
>
>
>
Can you tree 2.6.25? There are some fixes in that kernel related to
PCIe support:
[POWERPC] FSL: Rework PCI/PCIe support for 85xx/86xx
However, its odd that lspci doesn't even show the device. Are you
using u-boot? if so what version? and does u-boot see the LSI?
- k
___
Linuxppc-embedded mailing list
Linuxppc-embedded@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-embedded
Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip
Is the LSI chipset something you guys have on an add-on card
(something we can put into a reference board) or is hard wired into
your boards?
If its an add on card a pointer to one would be appreciated.
- k
On Jun 25, 2008, at 5:20 PM, Siva Prasad wrote:
I am also having problems with PCI-E device LSI 1064E on 2.6.24-rc6
kernel. This is running on 8641D processor.
I can see the device, access the config space, but seems access from
device to memory is not working. Same device (same exact hardware)
works
fine for 2.6.15 kernel.
- Siva
-Original Message-
Date: Wed, 25 Jun 2008 10:30:35 -0500
From: Kumar Gala <[EMAIL PROTECTED]>
Subject: Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI
1068 SAS chip
To: Vince Asbridge <[EMAIL PROTECTED]>
Cc: linuxppc-embedded@ozlabs.org
Message-ID: <[EMAIL PROTECTED]>
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed;
delsp=yes
On Jun 24, 2008, at 10:51 AM, Vince Asbridge wrote:
All,
I'm new to this mailing list, but have not had any luck finding
information on this issue.
Please be kind if I break the forum rules on my first post.
We recently tried to upgrade our Freescale CDS 8548 look-alike
module (code name ATCA1000) from the 2.6.11 based BSP to the 2.6.23
based BSP.
The upgrade went fairly smoothly, until we tried using SOME pci-e
devices (some work fine, some don't show up to lspci).
LSI pci-e controllers no longer show up at all!
We see the ixgbe (intel 10G), SiliconImage SATA controller but do
not see LSI devices (Specifically 1068 SAS, FC949-E fibrechannel).
We're guessing it's a resource issue behind the bridge, because the
LSI devices try to allocate 1 - 3M behind the bridge, but we can't
find the bug, or where we would debug such an issue.
The devices seem to "train" correctly, because we have an LED on the
pci-e switch (PLX 8 port pci-e switch), and it's ON indicating pci-e
link between the bridge and the 1068 device).
We're totally at a loss as to why this always worked on the 2.6.11
kernel but doesn?t work on 2.6.23.
Using lspci, the LSI adapters do not show up in the list at all, as
though they are not plugged into the system.
Is there something that needs to be done with respect to PCI-E
devices that is new in the 2.6.23 based BSP that did not need to be
done in the 2.6.11 based kit? For example, are pci resources
allocated by a different piece of code, that may have some issue
allocating resources for the LSI adapters?
Can you tree 2.6.25? There are some fixes in that kernel related to
PCIe support:
[POWERPC] FSL: Rework PCI/PCIe support for 85xx/86xx
However, its odd that lspci doesn't even show the device. Are you
using u-boot? if so what version? and does u-boot see the LSI?
- k
___
Linuxppc-embedded mailing list
Linuxppc-embedded@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-embedded
___
Linuxppc-embedded mailing list
Linuxppc-embedded@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-embedded
RE: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip
Kumar, We are using it onboard, not as add-on card. But I am sure there must be PCI-E to SAS PCI cards based on LSI SAS controllers. I just googled http://cgi.ebay.com/LSI-LSISAS3080X-R-8-Port-3Gb-s-SAS-PCI-X-Host-Bus-Ad apt_W0QQitemZ220243588314QQihZ012QQcategoryZ90715QQcmdZViewItem?refid=st ore - Siva -Original Message- From: Kumar Gala [mailto:[EMAIL PROTECTED] Sent: Thursday, June 26, 2008 6:37 AM To: Siva Prasad Cc: linuxppc-embedded@ozlabs.org Subject: Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip Is the LSI chipset something you guys have on an add-on card (something we can put into a reference board) or is hard wired into your boards? If its an add on card a pointer to one would be appreciated. - k On Jun 25, 2008, at 5:20 PM, Siva Prasad wrote: > I am also having problems with PCI-E device LSI 1064E on 2.6.24-rc6 > kernel. This is running on 8641D processor. > > I can see the device, access the config space, but seems access from > device to memory is not working. Same device (same exact hardware) > works > fine for 2.6.15 kernel. > > - Siva > > > -Original Message- > > Date: Wed, 25 Jun 2008 10:30:35 -0500 > From: Kumar Gala <[EMAIL PROTECTED]> > Subject: Re: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI > 1068 SASchip > To: Vince Asbridge <[EMAIL PROTECTED]> > Cc: linuxppc-embedded@ozlabs.org > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; > delsp=yes > > > On Jun 24, 2008, at 10:51 AM, Vince Asbridge wrote: > >> All, >> >> I'm new to this mailing list, but have not had any luck finding >> information on this issue. >> >> Please be kind if I break the forum rules on my first post. >> >> We recently tried to upgrade our Freescale CDS 8548 look-alike >> module (code name ATCA1000) from the 2.6.11 based BSP to the 2.6.23 >> based BSP. >> >> The upgrade went fairly smoothly, until we tried using SOME pci-e >> devices (some work fine, some don't show up to lspci). >> >> LSI pci-e controllers no longer show up at all! >> >> We see the ixgbe (intel 10G), SiliconImage SATA controller but do >> not see LSI devices (Specifically 1068 SAS, FC949-E fibrechannel). >> >> We're guessing it's a resource issue behind the bridge, because the >> LSI devices try to allocate 1 - 3M behind the bridge, but we can't >> find the bug, or where we would debug such an issue. >> >> The devices seem to "train" correctly, because we have an LED on the >> pci-e switch (PLX 8 port pci-e switch), and it's ON indicating pci-e >> link between the bridge and the 1068 device). >> >> We're totally at a loss as to why this always worked on the 2.6.11 >> kernel but doesn?t work on 2.6.23. >> >> Using lspci, the LSI adapters do not show up in the list at all, as >> though they are not plugged into the system. >> >> Is there something that needs to be done with respect to PCI-E >> devices that is new in the 2.6.23 based BSP that did not need to be >> done in the 2.6.11 based kit? For example, are pci resources >> allocated by a different piece of code, that may have some issue >> allocating resources for the LSI adapters? >> >> >> > Can you tree 2.6.25? There are some fixes in that kernel related to > PCIe support: > > [POWERPC] FSL: Rework PCI/PCIe support for 85xx/86xx > > However, its odd that lspci doesn't even show the device. Are you > using u-boot? if so what version? and does u-boot see the LSI? > > - k > > > ___ > Linuxppc-embedded mailing list > Linuxppc-embedded@ozlabs.org > https://ozlabs.org/mailman/listinfo/linuxppc-embedded ___ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded
RE: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS chip (solved!)
Solved! Problem summary. LSI logic devices (1068e, 1064e, fc949e) do not work with uBoot version 1.3.0 and Linux kernel 2.6.23, where they work perfectly fine in kernel 2.6.11. Symptom is that they do not show up at all to Linux once booted (lspci shows nothing). We obtained a pci-e analyzer and found the following: uBoot V1.3.0 now scans the pci configuration, and when it does it assigns bus numbers to the pci-e devices it finds. These numbers that are assigned are different from those that are assigned by Linux when the system boots. It is legal to change pci-e bus numbers "on the fly" but doing so requires a config write cycle to the pci-e device, because the pci-e spec states that the device must register a new bus number if a config write cycle is issues with the new bus number. When we boot the Freescale machine with uBoot 1.3.0 and kernel 2.6.23, the bus number under uBoot gets assigned to 1, and the bus number under Linux gets assigned to 4. Between the change of bus numbers we do not see a config write go to the LSI device with the new bus number, and therefore it continues to respond on bus 1 and ignore config reads to bus 4. We temporarily fixed the issue by defining the CONFIG_PCI_NOSCAN flag under uBoot, which causes the LSI device not to get assigned a bus number at uBoot, and therefore when Linux begins doing config reads at bus 4 to the LSI device registers the bus number and responds correctly. We will do a bit more investigation to see if this has already been fixed in a newer kernel, but for now not scanning at uBoot fixes the issue. It's a bit of a mystery why the LSI devices behave differently from any other pci-e device we put in the system, but they seem to be adhering to the letter of the specification. Vince Asbridge --- original post --- All, I'm new to this mailing list, but have not had any luck finding information on this issue. Please be kind if I break the forum rules on my first post. We recently tried to upgrade our Freescale CDS 8548 look-alike module (code name ATCA1000) from the 2.6.11 based BSP to the 2.6.23 based BSP. The upgrade went fairly smoothly, until we tried using SOME pci-e devices (some work fine, some don't show up to lspci). LSI pci-e controllers no longer show up at all! We see the ixgbe (intel 10G), SiliconImage SATA controller but do not see LSI devices (Specifically 1068 SAS, FC949-E fibrechannel). We're guessing it's a resource issue behind the bridge, because the LSI devices try to allocate 1 - 3M behind the bridge, but we can't find the bug, or where we would debug such an issue. The devices seem to "train" correctly, because we have an LED on the pci-e switch (PLX 8 port pci-e switch), and it's ON indicating pci-e link between the bridge and the 1068 device). We're totally at a loss as to why this always worked on the 2.6.11 kernel but doesn't work on 2.6.23. Using lspci, the LSI adapters do not show up in the list at all, as though they are not plugged into the system. Is there something that needs to be done with respect to PCI-E devices that is new in the 2.6.23 based BSP that did not need to be done in the 2.6.11 based kit? For example, are pci resources allocated by a different piece of code, that may have some issue allocating resources for the LSI adapters? I currently do not have access to a pci-e analyzer. Thanks for any help, Vince Asbridge <>___ Linuxppc-embedded mailing list Linuxppc-embedded@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-embedded
