Re: qemu_arm64_defconfig: PCI autoconfig fails for qemu-system-aarch64 -m 4G
Hi Heinrich, On Wed, 1 Jan 2020 at 19:58, Heinrich Schuchardt wrote: > > Dear all, > > I want to run qemu_arm64_defconfig with 4G. > > qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ > -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ > -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ > -device virtio-rng-pci Out of curiosity, why E1000 over virtio-net? > I see an error: > ... > pci_hose_phys_to_bus: invalid physical address > > The error does not occur for a smaller RAM size. > > I added some debug output: > > _dm_pci_phys_to_bus: > phys_addr 0x00013ffecb40, > res->phys_start 0x1000, > res->bus_start 0x1000 > > As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the > calculated bus address is truncated. Makes sense. > If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the > memory is less then 4 GiB an error > PCI: Failed autoconfig bar 10 > occurs. I believe this happens because the handling of the ranges DT property handling (or it could be in the PCI core itself, not sure) is a bit simplistic in U-Boot. The ranges property in QEMU-generated dtb contains two PCI_REGION_MEM ranges, one below 32-bit boundary and one above it. But decode_regions() in U-Boot only supports one range of given type. When CONFIG_SYS_PCI_64BIT is unset, only the region below 32-bit boundary is used, otherwise the one above 32-bits is used. See commit 52ba907328ec069ff1cec6cbe659f1714c68ed33. Thus with CONFIG_SYS_PCI_64BIT set any PCI devices which do not support 64-bit mem BARs stop working. This should happen regardless of the RAM size, right? > For 3GiB memory I observed the following values: > > _dm_pci_phys_to_bus: > phys_addr 0xfffecbc0, > res->phys_start 0x1000, > res->bus_start 0x1000 > > When I define type pci_addr_t as 64bit everything works correctly. > > Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT > and not on the bitness of the system? As I understand it, some PCI host controllers may require using 64-bit PCI bus addresses but still be connected to a 32-bit CPU. > It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented > at all. I wonder if Kumar remembers why he introduced it in 2008. > > I tried to change the definition of the following types in pci.h > > typedef phys_addr_t pci_addr_t; > typedef phys_addr_t pci_size_t; > > This lets qemu-x86_defconfig fail with > Error binding driver 'cpu_qemu': -12 > > Using > > typedef unsigned long pci_addr_t; > typedef unsigned long pci_size_t; > > does not produce the error but I am not sure if this is the right approach. > One solution would be to fix decode_regions() to work with multiple regions of the same type and then enable CONFIG_SYS_PCI_64BIT. I think another solution would be to simply limit U-Boot from being relocated above the 32-bit boundary. IIRC some other 64-bit boards do this as well. In fact, this might be the better solution because some bus mastering PCI devices cannot access memory above 32-bit. As far as I can tell, E1000 is not affected but for example EHCI is.
Re: qemu_arm64_defconfig: PCI autoconfig fails for qemu-system-aarch64 -m 4G
On 1/2/20 12:28 PM, Tuomas Tynkkynen wrote: Hi Heinrich, On Wed, 1 Jan 2020 at 19:58, Heinrich Schuchardt wrote: Dear all, I want to run qemu_arm64_defconfig with 4G. qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ -device virtio-rng-pci Out of curiosity, why E1000 over virtio-net? I see an error: ... pci_hose_phys_to_bus: invalid physical address The error does not occur for a smaller RAM size. I added some debug output: _dm_pci_phys_to_bus: phys_addr 0x00013ffecb40, res->phys_start 0x1000, res->bus_start 0x1000 As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the calculated bus address is truncated. Makes sense. If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the memory is less then 4 GiB an error PCI: Failed autoconfig bar 10 occurs. I believe this happens because the handling of the ranges DT property handling (or it could be in the PCI core itself, not sure) is a bit simplistic in U-Boot. The ranges property in QEMU-generated dtb contains two PCI_REGION_MEM ranges, one below 32-bit boundary and one above it. But decode_regions() in U-Boot only supports one range of given type. When CONFIG_SYS_PCI_64BIT is unset, only the region below 32-bit boundary is used, otherwise the one above 32-bits is used. See commit 52ba907328ec069ff1cec6cbe659f1714c68ed33. Thus with CONFIG_SYS_PCI_64BIT set any PCI devices which do not support 64-bit mem BARs stop working. This should happen regardless of the RAM size, right? For 3GiB memory I observed the following values: _dm_pci_phys_to_bus: phys_addr 0xfffecbc0, res->phys_start 0x1000, res->bus_start 0x1000 When I define type pci_addr_t as 64bit everything works correctly. Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT and not on the bitness of the system? As I understand it, some PCI host controllers may require using 64-bit PCI bus addresses but still be connected to a 32-bit CPU. It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented at all. I wonder if Kumar remembers why he introduced it in 2008. I tried to change the definition of the following types in pci.h typedef phys_addr_t pci_addr_t; typedef phys_addr_t pci_size_t; This lets qemu-x86_defconfig fail with Error binding driver 'cpu_qemu': -12 Using typedef unsigned long pci_addr_t; typedef unsigned long pci_size_t; does not produce the error but I am not sure if this is the right approach. One solution would be to fix decode_regions() to work with multiple regions of the same type and then enable CONFIG_SYS_PCI_64BIT. I think another solution would be to simply limit U-Boot from being relocated above the 32-bit boundary. IIRC some other 64-bit boards do this as well. In fact, this might be the better solution because some bus mastering PCI devices cannot access memory above 32-bit. As far as I can tell, E1000 is not affected but for example EHCI is. Hello Tuomas, thank you for your analysis. Changing the relocation address has amongst others the following side effects: * reducing the maximum memory for function tracing * reducing the available memory for UEFI Actually I was increasing RAM for QEMU due to working with function tracing. Where in decode_regions() do you see the limitation to work with multiple regions of the same type? Best regards Heinrich
qemu_arm64_defconfig: PCI autoconfig fails for qemu-system-aarch64 -m 4G
Dear all, I want to run qemu_arm64_defconfig with 4G. qemu-system-aarch64 -machine virt -m 4G -smp cores=2 \ -bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \ -netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \ -device virtio-rng-pci I see an error: No working controllers found e1000: 52:54:00:12:34:56 Warning: e1000#0 using MAC address from ROM BOOTP broadcast 1 pci_hose_phys_to_bus: invalid physical address BOOTP broadcast 2 pci_hose_phys_to_bus: invalid physical address The error does not occur for a smaller RAM size. I added some debug output: _dm_pci_phys_to_bus: phys_addr 0x00013ffecb40, res->phys_start 0x1000, res->bus_start 0x1000 As qemu_arm64_defconfig does not define CONFIG_SYS_PCI_64BIT the calculated bus address is truncated. If CONFIG_SYS_PCI_64BIT is defined for qemu_arm64_defconfig and the memory is less then 4 GiB an error PCI: Failed autoconfig bar 10 occurs. For 3GiB memory I observed the following values: _dm_pci_phys_to_bus: phys_addr 0xfffecbc0, res->phys_start 0x1000, res->bus_start 0x1000 When I define type pci_addr_t as 64bit everything works correctly. Why does the number of bits in pci_addr_t depend on CONFIG_SYS_PCI_64BIT and not on the bitness of the system? It is especially worrisome that CONFIG_SYS_PCI_64BIT is not documented at all. I wonder if Kumar remembers why he introduced it in 2008. I tried to change the definition of the following types in pci.h typedef phys_addr_t pci_addr_t; typedef phys_addr_t pci_size_t; This lets qemu-x86_defconfig fail with Error binding driver 'cpu_qemu': -12 Using typedef unsigned long pci_addr_t; typedef unsigned long pci_size_t; does not produce the error but I am not sure if this is the right approach. Best regards Heinrich
