https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231923
Bug ID: 231923
Summary: [pci] AMD Ryzen X370 chipset PCIe bridge failed to
allocate initial memory window
Product: Base System
Version: CURRENT
Hardware: amd64
OS: Any
Status: New
Severity: Affects Only Me
Priority: ---
Component: kern
Assignee: b...@freebsd.org
Reporter: greg@unrelenting.technology
Running fairly recent modified CURRENT (actually ALPHA7) from September 26.
(Haven't noticed PCIe related changes in the commits since then.)
Moved the system from a SATA SSD to an NVMe one. The Mellanox network card that
was installed in the bottom (connected to the X370 chipset) PCIe slot stopped
working:
pcib8: irq 32 at device 4.0 on pci3
pcib3: attempting to grow memory window for (0xfdf0-0xfe0f,0x20)
front candidate range: 0xfdf0-0xfe0f
pcib8: failed to allocate initial memory window: 0xfdf0-0xfe0f
pcib3: allocated prefetch range (0xf080-0xf0ff) for rid 24 of pcib8
pcib8: domain0
pcib8: secondary bus 36
pcib8: subordinate bus 36
pcib8: prefetched decode 0xf080-0xf0ff
pcib8: could not get PCI interrupt routing table for
\134_SB_.PCI0.GPP2.PT02.PT24 - AE_NOT_FOUND
pci8: on pcib8
pcib8: allocated bus range (36-36) for rid 0 of pci8
pci8: domain=0, physical bus=36
pcib3: attempting to grow memory window for (0xfe00-0xfe0f,0x10)
front candidate range: 0xfe00-0xfe0f
pcib8: failed to allocate initial memory window
(0xfe00-0xfe0f,0x10)
pci8: pci0:36:0:0 bar 0x10 failed to allocate
map[18]: type Prefetchable Memory, range 64, base 0xf080, size 23,
memory disabled
pcib8: allocated prefetch range (0xf080-0xf0ff) for rid 18 of
pci0:36:0:0
pcib2: matched entry for 3.0.INTA
pcib2: slot 0 INTA hardwired to IRQ 32
pcib3: slot 4 INTA is routed to irq 32
pcib8: slot 0 INTA is routed to irq 32
pci8: at device 0.0 (no driver attached)
[...]
mlx4_core0: mem 0xf080-0xf0ff irq 32 at device 0.0 on pci8
mlx4_core: Mellanox ConnectX core driver v3.4.1 (October 2017)
mlx4_core: Initializing mlx4_core
pcib3: attempting to grow memory window for (0-0x,0x10)
front candidate range: 0xfe10-0xfe1f
back candidate range: 0xfe30-0xfe3f
pcib8: failed to allocate initial memory window (0-0x,0x10)
mlx4_core0: 0x10 bytes of rid 0x10 res 3 failed (0, 0x).
mlx4_core0: Couldn't get PCI resources, aborting
device_attach: mlx4_core0 attach returned 22
The same thing is happening to pcib9, which has one of the XHCI controllers on
it (so some USB3 ports aren't working), but that's been happening even before
the NVMe drive:
xhci2: irq 32 at device 0.0 on pci9
pcib3: attempting to grow memory window for (0-0x,0x10)
front candidate range: 0xfe10-0xfe1f
back candidate range: 0xfe30-0xfe3f
pcib9: failed to allocate initial memory window (0-0x,0x8000)
xhci2: 0x8000 bytes of rid 0x10 res 3 failed (0, 0x).
xhci2: Could not map memory
device_attach: xhci2 attach returned 12
pciconf looks like this:
pcib8@pci0:22:4:0: class=0x060400 card=0x33061b21 chip=0x43b41022 rev=0x02
hdr=0x01
vendor = 'Advanced Micro Devices, Inc. [AMD]'
device = '300 Series Chipset PCIe Port'
class = bridge
subclass = PCI-PCI
bus range = 36-36
window[1c] = type I/O Port, range 32, addr 0xfff000-0xfff, disabled
window[20] = type Memory, range 32, addr 0xfff0-0xf, disabled
window[24] = type Prefetchable Memory, range 64, addr
0xf080-0xf0ff, enabled
cap 05[50] = MSI supports 1 message, 64 bit
cap 01[78] = powerspec 3 supports D0 D3 current D0
cap 10[80] = PCI-Express 2 downstream port max data 128(512) RO NS
link x4(x4) speed 5.0(5.0) ASPM disabled(L0s/L1)
slot 1 power limit 26000 mW
cap 0d[c0] = PCI Bridge card=0x33061b21
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
ecap 0019[200] = PCIe Sec 1 lane errors 0
ecap 001e[400] = unknown 1
Corrected = Receiver Error
none1@pci0:36:0:0: class=0x02 card=0x002115b3 chip=0x675015b3 rev=0xb0
hdr=0x00
vendor = 'Mellanox Technologies'
device = 'MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s]'
class = network
subclass = ethernet
bar [10] = type Memory, range 64, base 0xfe00, size 1048576, disabled
bar [18] = type Prefetchable Memory, range 64, base 0xf080, size
8388608, disabled
cap 01[40] = powerspec 3 supports D0 D3 current D0
cap 03[48] = VPD
cap 11[9c] = MSI-X supports 128 messages
Table in map 0x10[0x7c000], PBA in map 0x10[0x7d000]
cap 10[60] = PCI-Express 2 endpoint max data 128(256) FLR
