Hello! On Sunday 07 November 2021 18:08:56 Tony Dinh wrote: > Hi Pali, > > On Sun, Nov 7, 2021 at 4:58 PM Tony Dinh <mibo...@gmail.com> wrote: > > > > Hi Pali, > > > > Looks like it is working! But in a weird way. Please see my responses > > in between, and also in the test log below. > > > > On Sun, Nov 7, 2021 at 3:45 PM Pali Rohár <p...@kernel.org> wrote: > > > > > > Hello! > > > > > > On Sunday 07 November 2021 12:56:37 Tony Dinh wrote: > > > > Hi Pali, > > > > > > > > I've applied your patch, and ran some tests. Looks like we got to the > > > > bind, but it was not probing (or the probe died silently). Later, I > > > > tried pci enum and got some errors during probing. > > > > > > It looks like that arch/arm/mach-kirkwood/cpu.c already setup PCIe > > > windows. Therefore when pci_mvebu.c tries it too, it fails on error > > > "conflicts with another window" which you see in the log. > > > > > > Could you try to comment calling of "mvebu_mbus_add_window_by_id" > > > function in pci_mvebu.c? As PCIe window setup should be done exactly > > > once. > > > > > > Also try to call 'pci 0' and 'pci 1' after the 'pci enum'. > > > ... > > > > > > > > - These are relevant configs used in the build (not sure if I need to > > > > use CONFIG_DM_PCI_COMPAT?) > > > > > > > > CONFIG_DM=y > > > > CONFIG_CMD_PCI=y > > > > CONFIG_PCI=y > > > > CONFIG_PCI_MVEBU=y > > > > CONFIG_PCI_ENDPOINT=y > > > > CONFIG_DM_PCI_COMPAT=y > > > > CONFIG_USB_XHCI_HCD=y > > > > CONFIG_USB_XHCI_PCI=y > > > > CONFIG_USB_XHCI_MVEBU=y > > > > > > Following options should be enough: > > > > > > CONFIG_CMD_PCI=y > > > CONFIG_PCI=y > > > # CONFIG_DM_PCI_COMPAT is not set > > > CONFIG_PCI_PNP=y > > > CONFIG_PCI_MVEBU=y > > > > Since the PCI bus in this box is used for USB 3.0, I've included > > various XHCI configs to see if I can probe this USB 3.0 flash drive > > also (that's the ultimate test).
OK! > > > > > > Option CONFIG_PCI_PNP=y is required to see endpoint cards. Without it > > > U-Boot would see only PCIe Root Ports. But for first basic tests it > > > should be enough. > > > > > > > BTW, the topic is no longer kwboot, should we move this to another new > > > > thread? i.e. Testing PCI MVEBU with Kirkwood SoCs. > > > > > > Changed :-) > > > > Thanks :) > > > > Here is the log of a successful spin up for the PCI bus and USB 3.0 > > drive. Please see my commentary ### in between the logs. > > > > U-Boot 2022.01-rc1-00054-g52207514ba-dirty (Nov 07 2021 - 16:03:46 -0800) > > Pogoplug V4 > > > > SoC: Kirkwood 88F6281_A1 > > DRAM: 128 MiB > > device_probe: > > device_probe: exit 0 > > > > > > ### The device_probe above seems to have nothing happening. > > > > > > mvebu_pcie_bind: begin > > mvebu_pcie_bind: got an available node > > Bound device to pcie@82000000 > > mvebu_pcie_bind: bound > > mvebu_pcie_bind: exit 0 > > Bound device pcie@82000000 to mbus@f1000000 > > Bound device mbus@f1000000 to root_driver > > Bound device ehci@50000 to ocp@f1000000 > > Bound device ethernet-controller@72000 to ocp@f1000000 > > Bound device sata@80000 to ocp@f1000000 > > Bound device mvs...@90000.blk to mvsdio@90000 > > Bound device mvsdio@90000 to ocp@f1000000 > > Bound device ocp@f1000000 to root_driver > > NAND: 128 MiB > > MMC: device_probe: > > device_probe: > > device_probe: > > device_probe: exit 0 > > device_probe: exit 0 > > device_probe: > > device_probe: > > mvsdio@90000: 0 > > Loading Environment from NAND... *** Warning - bad CRC, using default > > environment > > > > In: serial > > Out: serial > > Err: serial > > Net: device_probe: > > device_probe: > > > > Warning: ethernet-controller@72000 (eth0) using random MAC address - > > 86:b8:84:ee:a9:a5 > > device_probe: exit 0 > > eth0: ethernet-controller@72000 > > Hit any key to stop autoboot: 0 > > Pogo_V4> usb reset > > resetting USB... > > Bus ehci@50000: device_probe: > > device_probe: > > USB EHCI 1.00 > > device_probe: exit 0 > > scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000 > > device_probe: > > device_probe: > > Bound device usb_hub to usb_hub > > device_probe: > > device_probe: > > device_probe: exit 0 > > Bound device usb_mass_storage to usb_hub > > device_probe: > > device_probe: > > Bound device usb_mass_storage.lun0 to usb_mass_storage > > device_probe: exit 0 > > device_probe: exit 0 > > 3 USB Device(s) found > > scanning usb for storage devices... 1 Storage Device(s) found > > > > > > ### Try to bring up both USB drives (2.0 and 3.0). But only the USB > > 2.0 was up as shown above. > > > > > > Pogo_V4> pci enum > > device_probe: > > device_probe: > > device_probe: > > device_probe: > > device_probe: exit 0 > > device_probe: exit 0 > > mvebu_pcie_probe: > > Cannot add window '4:e8', conflicts with another window > > PCIe unable to add mbus window for mem at 90000000+10000000 > > Cannot add window '4:e0', conflicts with another window > > PCIe unable to add mbus window for IO at c0000000+00010000 Error is still there... Have you tried to comment and disable calling "mvebu_mbus_add_window_by_id" function from pci_mvebu.c? > > mvebu_pcie_probe: exit 0 > > Bound device pci_0:0.0 to pcie0.0 > > device_probe: > > device_probe: > > Bound device xhci_pci to pci_0:0.0 > > device_probe: exit 0 > > device_probe: exit 0 > > device_probe: > > > > ### So I did the pci enum above to get the XHCI bound to PCIe. > > > > Pogo_V4> usb reset > > resetting USB... > > Bus ehci@50000: device_probe: > > device_probe: > > USB EHCI 1.00 > > device_probe: exit 0 > > Bus xhci_pci: device_probe: > > device_probe: > > Register 400081f NbrPorts 4 > > Starting the controller > > USB XHCI 1.00 > > device_probe: exit 0 > > scanning bus ehci@50000 for devices... Bound device usb_hub to ehci@50000 > > device_probe: > > device_probe: > > Bound device usb_hub to usb_hub > > device_probe: > > device_probe: > > device_probe: exit 0 > > Bound device usb_mass_storage to usb_hub > > device_probe: > > device_probe: > > Bound device usb_mass_storage.lun0 to usb_mass_storage > > device_probe: exit 0 > > device_probe: exit 0 > > 3 USB Device(s) found > > scanning bus xhci_pci for devices... Bound device usb_hub to xhci_pci > > device_probe: > > device_probe: > > Bound device usb_mass_storage to usb_hub > > device_probe: > > device_probe: > > Bound device usb_mass_storage.lun0 to usb_mass_storage > > device_probe: exit 0 > > device_probe: exit 0 > > 2 USB Device(s) found > > scanning usb for storage devices... 2 Storage Device(s) found > > > > > > ### Now both drives are up! > > > > > > Pogo_V4> pci 0 > > device_probe: > > Scanning PCI devices on bus 0 > > BusDevFun VendorId DeviceId Device Class Sub-Class > > _____________________________________________________________ > > 00.00.00 0x11ab 0x6281 Bridge device 0x04 > > Pogo_V4> pci 1 > > device_probe: > > Scanning PCI devices on bus 1 > > BusDevFun VendorId DeviceId Device Class Sub-Class > > _____________________________________________________________ > > 01.00.00 0x1b73 0x1009 Serial bus controller 0x03 > > > > > > ### And the pci command has meaningful output at this point as shown above. > > > > > > Pogo_V4> usb tree > > USB device tree: > > 1 Hub (480 Mb/s, 0mA) > > | u-boot EHCI Host Controller > > | > > +-2 Hub (480 Mb/s, 100mA) > > | GenesysLogic USB2.0 Hub > > | > > +-3 Mass Storage (480 Mb/s, 224mA) > > SanDisk Ultra Fit 4C531001560827107320 > > > > 1 Hub (5 Gb/s, 0mA) > > | U-Boot XHCI Host Controller > > | > > +-2 Mass Storage (5 Gb/s, 224mA) > > SanDisk Cruzer Glide 3.0 4C530000130418116112 > > > > Pogo_V4> usb info > > 1: Hub, USB Revision 2.0 > > - u-boot EHCI Host Controller > > - Class: Hub > > - PacketSize: 64 Configurations: 1 > > - Vendor: 0x0000 Product 0x0000 Version 1.0 > > Configuration: 1 > > - Interfaces: 1 Self Powered 0mA > > Interface: 0 > > - Alternate Setting 0, Endpoints: 1 > > - Class Hub > > - Endpoint 1 In Interrupt MaxPacket 8 Interval 255ms > > > > 2: Hub, USB Revision 2.0 > > - GenesysLogic USB2.0 Hub > > - Class: Hub > > - PacketSize: 64 Configurations: 1 > > - Vendor: 0x05e3 Product 0x0610 Version 144.48 > > Configuration: 1 > > - Interfaces: 1 Self Powered Remote Wakeup 100mA > > Interface: 0 > > - Alternate Setting 0, Endpoints: 1 > > - Class Hub > > - Endpoint 1 In Interrupt MaxPacket 1 Interval 12ms > > - Endpoint 1 In Interrupt MaxPacket 1 Interval 12ms > > > > 3: Mass Storage, USB Revision 2.10 > > - SanDisk Ultra Fit 4C531001560827107320 > > - Class: (from Interface) Mass Storage > > - PacketSize: 64 Configurations: 1 > > - Vendor: 0x0781 Product 0x5583 Version 1.0 > > Configuration: 1 > > - Interfaces: 1 Bus Powered 224mA > > Interface: 0 > > - Alternate Setting 0, Endpoints: 2 > > - Class Mass Storage, Transp. SCSI, Bulk only > > - Endpoint 1 In Bulk MaxPacket 512 > > - Endpoint 2 Out Bulk MaxPacket 512 > > > > 1: Hub, USB Revision 3.0 > > - U-Boot XHCI Host Controller > > - Class: Hub > > - PacketSize: 512 Configurations: 1 > > - Vendor: 0x0000 Product 0x0000 Version 1.0 > > Configuration: 1 > > - Interfaces: 1 Self Powered 0mA > > Interface: 0 > > - Alternate Setting 0, Endpoints: 1 > > - Class Hub > > - Endpoint 1 In Interrupt MaxPacket 8 Interval 255ms > > > > 2: Mass Storage, USB Revision 3.0 > > - SanDisk Cruzer Glide 3.0 4C530000130418116112 > > - Class: (from Interface) Mass Storage > > - PacketSize: 512 Configurations: 1 > > - Vendor: 0x0781 Product 0x5597 Version 1.0 > > Configuration: 1 > > - Interfaces: 1 Bus Powered 224mA > > Interface: 0 > > - Alternate Setting 0, Endpoints: 2 > > - Class Mass Storage, Transp. SCSI, Bulk only > > - Endpoint 1 In Bulk MaxPacket 1024 > > - Endpoint 2 Out Bulk MaxPacket 1024 > > > > ### The XHCI Host Controller and the SanDisk Cruzer Glide are now > > active as shown above. > > > > ### So list the files on this SanDisk Cruzer Glide to make sure we can read > > it. > > > > > > Pogo_V4> ext2ls usb 1:1 / > > device_probe: > > device_probe: > > device_probe: exit 0 > > <DIR> 4096 . > > <DIR> 4096 .. > > 522666 uboot.2021.07-tld-1.pogo_v4.bin > > 524288 uboot.2021.07-tld-1.pogo_v4.mtd0.kwb > > > > Great works Pali! > > > > I think some more investigation is needed. Why did we need to do "pci > > enum", and then "usb reset", in that order, to get the PCI bus and the > > XHCI controller probed? there must be something missing in the process > > somewhere between the Device uclass, the PCI uclass, and the pci_mvebu > > uclass? > > I think I can see the order of enumeration. PCI must be enumerated > first, and then XHCI being the controller on this host bus would come > alive? I think we can live with 'pci enum' and 'usb reset' to get the > USB 3.0 drives enumerated. However, it seems just a little bit > unintuitive. 'pci enum' should be called internally by U-Boot during loading. So only 'usb start' would be needed. But from your boot log it looks like that PCI enumaration was not done and so calling 'pci enum' manually is needed. I will look into U-Boot code why it happens... Anyway, based on your test, PCIe must work correctly :) Could you send config space dump of PCIe Root Port? Following U-Boot command prints it on terminal: 'pci display.b 0.0.0 0 200'
