Hi Pali,
I've tried the test with mdelay(200) at the end of mvebu_pcie_probe().
Please see below.
On Tue, Nov 9, 2021 at 7:17 PM Tony Dinh <mibo...@gmail.com> wrote:
>
> Hi Pali,
>
> On Tue, Nov 9, 2021 at 3:05 PM Pali Rohár <p...@kernel.org> wrote:
> >
> > Hello!
> >
> > On Tuesday 09 November 2021 14:51:33 Tony Dinh wrote:
> > > Hi Pali,
> > >
> > > On Tue, Nov 9, 2021 at 7:08 AM Pali Rohár <p...@kernel.org> wrote:
> > > >
> > > > On Monday 08 November 2021 22:34:51 Tony Dinh wrote:
> > > > > The above log was the build with the following configs:
> > > > > CONFIG_CMD_PCI=y
> > > > > CONFIG_PCI=y
> > > > > CONFIG_PCI_MVEBU=y
> > > > > CONFIG_PCI_PNP=y
> > > > > CONFIG_USB_XHCI_HCD=y
> > > > > CONFIG_USB_XHCI_PCI=y
> > > > > CONFIG_USB_XHCI_MVEBU=y
> > > > > CONFIG_PCI_INIT_R=y
> > > > > CONFIG_SYS_EARLY_PCI_INIT=y
> > > > >
> > > > >
> > > > > Any idea how to debug this? there must be some differences between the
> > > > > pci_init call early on, and much later with "pci enum".
> > > >
> > > > Hello! The only difference is when pci_init() function is called. So it
> > > > looks like that it needs to be called later (perhaps because PCIe needs
> > > > some other steps for initialization?).
> > > >
> > > > Could you try to disable CONFIG_SYS_EARLY_PCI_INIT option if it helps?
> > >
> > > Please correct me if I'm wrong. It is a compound condition in
> > > board_r.c, so it would not be executed anyway?
> > >
> > > #if defined(CONFIG_PCI_INIT_R) && defined(CONFIG_SYS_EARLY_PCI_INIT)
> > >
> >
> > And few lines below is section:
> >
> > #if defined(CONFIG_PCI_INIT_R) && !defined(CONFIG_SYS_EARLY_PCI_INIT)
> > /*
> > * Do pci configuration
> > */
> > pci_init,
> > #endif
> >
> > So CONFIG_SYS_EARLY_PCI_INIT controls if pci_init() is called earlier or
> > later.
> >
> > > > And could you try to put pci_init() into board_early_init_r() and
> > > > disable both CONFIG_SYS_EARLY_PCI_INIT and CONFIG_PCI_INIT_R if it
> > > > helps?
> > >
> > > This did not help. I've added board_early_init_r to the board file,
> > > and called pci_init() there. In the debug output, I can see pci_init()
> > > executed early. But later, "usb start" still hangs at EHCI enumeration
> > > like before.
> > >
> > > scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000
> > >
> > > Strangely, on a hunch, I ran it a second time and then did "pci enum"
> > > and "usb start", I got a bit further. Now it scans the XHCI bus, and
> > > gets 3 USB devices, but then hangs.
> > >
> > > scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000
> > > Bound device usb_hub to usb_hub
> > > Bound device usb_mass_storage to usb_hub
> > > Bound device usb_mass_storage.lun0 to usb_mass_storage
> > > 3 USB Device(s) found
> > > scanning bus xhci_pci for devices... Bound device usb_hub to xhci_pci
> > >
> > > So it seems that after the board has finished initialization, calling
> > > pci_init() again from "pci enum" will result in more probing
> > > processing.
> >
> > So it means that there are some timing issues.
> >
> > Could you try to add "mdelay(200);" at the end of mvebu_pcie_probe()
> > function? For mdelay you probably need also "#include <linux/delay.h>".
I've tried the mdelay(200) at the end of mvebu_pcie_probe(). In this
test I set the configs is back to:
CONFIG_CMD_PCI=y
CONFIG_PCI=y
CONFIG_PCI_MVEBU=y
CONFIG_PCI_PNP=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_PCI=y
CONFIG_USB_XHCI_MVEBU=y
CONFIG_PCI_INIT_R=y
CONFIG_SYS_EARLY_PCI_INIT=y
The behavior is the same as before. It hangs at "usb start".
U-Boot 2022.01-rc1-00054-g52207514ba-dirty (Nov 10 2021 - 13:56:09 -0800)
Pogoplug V4
SoC: Kirkwood 88F6192_A1
DRAM: 128 MiB
device_probe:
device_probe: exit 0
mvebu_pcie_bind: begin
mvebu_pcie_bind: got an available node
Bound device to pcie@82000000
mvebu_pcie_bind: bound
mvebu_pcie_bind: exit 0
Bound device pcie@82000000 to mbus@f1000000
Bound device mbus@f1000000 to root_driver
Bound device ehci@50000 to ocp@f1000000
Bound device ethernet-controller@72000 to ocp@f1000000
Bound device sata@80000 to ocp@f1000000
Bound device mvs...@90000.blk to mvsdio@90000
Bound device mvsdio@90000 to ocp@f1000000
Bound device ocp@f1000000 to root_driver
NAND: 128 MiB
MMC: device_probe:
device_probe:
device_probe:
device_probe: exit 0
device_probe: exit 0
device_probe:
device_probe:
mvsdio@90000: 0
Loading Environment from NAND... *** Warning - bad CRC, using default
environment
pci_init:
device_probe:
device_probe:
device_probe:
device_probe:
device_probe: exit 0
device_probe: exit 0
mvebu_pcie_probe:
mvebu_pcie_probe: exit 0
Bound device pci_0:0.0 to pcie0.0
device_probe:
device_probe:
Bound device xhci_pci to pci_0:0.0
device_probe: exit 0
device_probe: exit 0
device_probe:
pci_init: exit 0
In: serial
Out: serial
Err: serial
Net: device_probe:
device_probe:
Warning: ethernet-controller@72000 (eth0) using random MAC address -
a2:f2:2b:f9:74:ac
device_probe: exit 0
eth0: ethernet-controller@72000
Hit any key to stop autoboot: 0
Pogo_V4> pci 0
device_probe:
Scanning PCI devices on bus 0
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
00.00.00 0x11ab 0x6281 Bridge device 0x04
Pogo_V4> pci 1
device_probe:
Scanning PCI devices on bus 1
BusDevFun VendorId DeviceId Device Class Sub-Class
_____________________________________________________________
01.00.00 0x1b73 0x1009 Serial bus controller 0x03
Pogo_V4> usb start
starting USB...
Bus ehci@50000: device_probe:
device_probe:
USB EHCI 1.00
device_probe: exit 0
Bus xhci_pci: device_probe:
device_probe:
Register 400081f NbrPorts 4
Starting the controller
USB XHCI 1.00
device_probe: exit 0
scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000
device_probe:
device_probe:
I've also tried mdelay(3000), just to be sure. The result was the same
hang in 'usb start'.
Thanks,
Tony
> >
>
> I have not tried this, given we are successful with board_late_init now.
>
> > Or could you try to add "int board_late_init(void) { pci_init(); return 0;
> > }"
> > into your board code?
>
> board_late_init is working! Here is the log.
>
>
> U-Boot 2022.01-rc1-00054-g52207514ba-dirty (Nov 09 2021 - 18:28:37 -0800)
> Pogoplug V4
>
> SoC: Kirkwood 88F6192_A1
> DRAM: 128 MiB
> device_probe:
> device_probe: exit 0
> mvebu_pcie_bind: begin
> mvebu_pcie_bind: got an available node
> Bound device to pcie@82000000
> mvebu_pcie_bind: bound
> mvebu_pcie_bind: exit 0
> Bound device pcie@82000000 to mbus@f1000000
> Bound device mbus@f1000000 to root_driver
> Bound device ehci@50000 to ocp@f1000000
> Bound device ethernet-controller@72000 to ocp@f1000000
> Bound device sata@80000 to ocp@f1000000
> Bound device mvs...@90000.blk to mvsdio@90000
> Bound device mvsdio@90000 to ocp@f1000000
> Bound device ocp@f1000000 to root_driver
> NAND: 128 MiB
> MMC: device_probe:
> device_probe:
> device_probe:
> device_probe: exit 0
> device_probe: exit 0
> device_probe:
> device_probe:
> mvsdio@90000: 0
> Loading Environment from NAND... *** Warning - bad CRC, using default
> environment
>
> In: serial
> Out: serial
> Err: serial
> pci_init:
> device_probe:
> device_probe:
> device_probe:
> device_probe:
> device_probe: exit 0
> device_probe: exit 0
> mvebu_pcie_probe:
> mvebu_pcie_probe: exit 0
> Bound device pci_0:0.0 to pcie0.0
> device_probe:
> device_probe:
> Bound device xhci_pci to pci_0:0.0
> device_probe: exit 0
> device_probe: exit 0
> device_probe:
> pci_init: exit 0
> Net: device_probe:
> device_probe:
>
> Warning: ethernet-controller@72000 (eth0) using random MAC address -
> 4a:8f:e6:9d:8b:79
> device_probe: exit 0
> eth0: ethernet-controller@72000
> Hit any key to stop autoboot: 0
>
> Pogo_V4> pci 0
> device_probe:
> Scanning PCI devices on bus 0
> BusDevFun VendorId DeviceId Device Class Sub-Class
> _____________________________________________________________
> 00.00.00 0x11ab 0x6281 Bridge device 0x04
> Pogo_V4> pci 1
> device_probe:
> Scanning PCI devices on bus 1
> BusDevFun VendorId DeviceId Device Class Sub-Class
> _____________________________________________________________
> 01.00.00 0x1b73 0x1009 Serial bus controller 0x03
>
> Pogo_V4> usb start
> starting USB...
> Bus ehci@50000: device_probe:
> device_probe:
> USB EHCI 1.00
> device_probe: exit 0
> Bus xhci_pci: device_probe:
> device_probe:
> Register 400081f NbrPorts 4
> Starting the controller
> USB XHCI 1.00
> device_probe: exit 0
> scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000
> device_probe:
> device_probe:
> Bound device usb_hub to usb_hub
> device_probe:
> device_probe:
> device_probe: exit 0
> Bound device usb_mass_storage to usb_hub
> device_probe:
> device_probe:
> Bound device usb_mass_storage.lun0 to usb_mass_storage
> device_probe: exit 0
> device_probe: exit 0
> 3 USB Device(s) found
> scanning bus xhci_pci for devices... Bound device usb_hub to xhci_pci
> device_probe:
> device_probe:
> Bound device usb_mass_storage to usb_hub
> device_probe:
> device_probe:
> Bound device usb_mass_storage.lun0 to usb_mass_storage
> device_probe: exit 0
> device_probe: exit 0
> 2 USB Device(s) found
> scanning usb for storage devices... 2 Storage Device(s) found
>
> Pogo_V4> usb tree
> USB device tree:
> 1 Hub (480 Mb/s, 0mA)
> | u-boot EHCI Host Controller
> |
> +-2 Hub (480 Mb/s, 100mA)
> | GenesysLogic USB2.0 Hub
> |
> +-3 Mass Storage (480 Mb/s, 224mA)
> SanDisk Ultra Fit 4C531001560827107320
>
> 1 Hub (5 Gb/s, 0mA)
> | U-Boot XHCI Host Controller
> |
> +-2 Mass Storage (5 Gb/s, 224mA)
> SanDisk Cruzer Glide 3.0 4C530000130418116112
>
> I think we got the problem identified! Most of other boards, the
> pci_init is called in early_init. Perhaps Kirwood is different. or
> perhaps there is some timing problem lurking in there somewhere?. I
> will try this new patch with the Zyxel NSA325 u-boot to see if we can
> observe the same behavior.
>
> And thanks for all the great work! I'll submit patches to add support
> for this Pogoplug V4 board if you think we are done enough for now.
> This board is a really good vehicle for Kirkwood SoCs testing (SATA,
> USB, PCIe, MMC, Gbit Ethernet), even more than the Sheevaplug can do.
>
> Thanks,
> Tony
>
>
> > In every test ensure that pci_init() is called only once. So if adding
> > pci_init() in board code then disable CONFIG_PCI_INIT_R.
