>..."Strangely, however, prtdiag sees the slots as populated:"...
I don't understand how it could be pci_boot/create_bus issues; cause pci_boot is functioning and allocating the buses *correctly* to relevant slots. As stated, Left-hand has 4::x8-PCI-e slots (2-cpu1, 2-cpu2) and right-hand has 2::x8-PCI-e slots (2-cpu2), which we see through prtdiag details about slots (1-to-6) assignments. ..which means socket 00 is working correctly, and socket 01 has some fault!!. surprising, very much nailed correctly relating to QPI interconnect issues. i guess PCH/C6xxx-chipset and motherboard specs need to be reviewed. prtpicl should display correctly details to resource addressing, cause slots are assigned. >..."The problem appears to be that Illumos doesn't recognize any PCI bridges beyond those attached to CPU1. yes, maybe its a PCIe-PCI bridging issue, even i doubt. try kdb debug state... ::bp pcieb`pcieb_attach :c pcieb`pcieb_dbg_print/W 0x1 --- this should display basic debugged information :c also. try with disabled (both) MSI and workaround patch to 41210/chipset individually. pcieb`pcieb_enable_msi/W 0x1 pcieb_disable_41210_wkarnd/W 0x1 i'm not sure if there is any resource conflict issue; mdb -k | ::interrupts as obviously, h/w is remote! ----------------------------------- Aakash Saini On Wed, May 1, 2013 at 6:54 AM, Garrett D'Amore <[email protected]> wrote: > Check the value of pci_bios_maxbus. > > I have a sneaking feeling that the problem may be related to acpi > enumeration of the bus. We are limited to seeing the busses that ACPI > properly enumerates. Take a look at the code in pci_boot.c for more > details. You should especially note the call sites for > create_pcie_root_bus. > > I'd love to help further, but lack of access to hardware that exhibits > this problem prevents me from doing any active debug myself. > > - Garrett > > On May 1, 2013, at 12:50 AM, Sašo Kiselkov <[email protected]> wrote: > > > Just a follow up on this: this appears to be an Illumos bug, though I > > haven't figured out how to resolve it yet. The problem appears to be > > that Illumos doesn't recognize any PCI bridges beyond those attached to > > CPU1. Notably, nothing on CPU2 appears to work. I've tested with Linux > > 3.5 and that works flawless, attached are the lspci -vvvv outputs. > > > > The problem appears to be that Illumos stops after PCI address 0b:00.0 > > (on-board MegaRAID SAS 2208). After that the scan should proceed with > > the QPI inter-processor links, but doesn't. > > > > The full lspci should look something like this: > > > > 00:00.0 Host bridge: Intel Corporation Xeon E5/Core i7 DMI.. > > 00:01.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:02.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:02.2 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:02.3 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:03.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:03.2 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 00:04.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.5 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:04.7 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:05.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:05.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 00:05.4 PIC: Intel Corporation Xeon E5/Core i7 I/O APIC (r.. > > 00:11.0 PCI bridge: Intel Corporation C600/X79 series chip.. > > 00:16.0 Communication controller: Intel Corporation C600/X.. > > 00:16.1 Communication controller: Intel Corporation C600/X.. > > 00:1a.0 USB controller: Intel Corporation C600/X79 series .. > > 00:1d.0 USB controller: Intel Corporation C600/X79 series .. > > 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (re.. > > 00:1f.0 ISA bridge: Intel Corporation C600/X79 series chip.. > > 00:1f.2 SATA controller: Intel Corporation C600/X79 series.. > > 00:1f.3 SMBus: Intel Corporation C600/X79 series chipset S.. > > 00:1f.6 Signal processing controller: Intel Corporation C6.. > > 02:00.0 Serial Attached SCSI controller: Intel Corporation.. > > 03:00.0 SCSI storage controller: LSI Logic / Symbios Logic.. > > 05:00.0 Ethernet controller: Intel Corporation 82599EB 10-.. > > 05:00.1 Ethernet controller: Intel Corporation 82599EB 10-.. > > 07:00.0 PCI bridge: Tundra Semiconductor Corp. Device 8113.. > > 08:01.0 VGA compatible controller: Matrox Electronics Syst.. > > 09:00.0 Ethernet controller: Intel Corporation I350 Gigabi.. > > 09:00.1 Ethernet controller: Intel Corporation I350 Gigabi.. > > 0b:00.0 RAID bus controller: LSI Logic / Symbios Logic Meg.. > > <<< Illumos gives up here, Linux continues >>> > > 7f:08.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:08.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:08.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:09.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:09.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:09.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0a.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0a.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0a.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0a.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0b.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0b.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0c.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0c.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0c.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0c.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0c.7 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0d.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0d.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0d.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0d.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0e.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0e.1 Performance counters: Intel Corporation Xeon E5/Co.. > > 7f:0f.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.5 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:0f.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.5 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:10.7 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:11.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:13.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 7f:13.1 Performance counters: Intel Corporation Xeon E5/Co.. > > 7f:13.4 Performance counters: Intel Corporation Xeon E5/Co.. > > 7f:13.5 Performance counters: Intel Corporation Xeon E5/Co.. > > 7f:13.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:00.0 PCI bridge: Intel Corporation Xeon E5/Core i7 DMI2.. > > 80:01.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 80:02.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 80:02.2 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 80:03.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 80:03.2 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO .. > > 80:04.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.1 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.3 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.4 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.5 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.6 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:04.7 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:05.0 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:05.2 System peripheral: Intel Corporation Xeon E5/Core .. > > 80:05.4 PIC: Intel Corporation Xeon E5/Core i7 I/O APIC (r.. > > 83:00.0 SCSI storage controller: LSI Logic / Symbios Logic.. > > > > See http://217.75.71.10/MB.png for the system interconnection diagram. > > Can somebody please recommend how to debug this issue? > > > > Cheers, > > -- > > Saso > > > > > > ------------------------------------------- > > illumos-discuss > > Archives: https://www.listbox.com/member/archive/182180/=now > > RSS Feed: > https://www.listbox.com/member/archive/rss/182180/22003744-9012f59c > > Modify Your Subscription: https://www.listbox.com/member/?&; > > Powered by Listbox: http://www.listbox.com > > > > ------------------------------------------- > illumos-developer > Archives: https://www.listbox.com/member/archive/182179/=now > RSS Feed: > https://www.listbox.com/member/archive/rss/182179/24096868-ef9d90b8 > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com > ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
