I instrumented detect_extended_topology() and ran again with 4 CPUs.
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswi...@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Mon Nov 23
07:18:36 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x0
(XEN) detect_extended_topology initial_apicid=0 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x0
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.632 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e7f90 -> ffff82d0802e9244
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=4 num_processors=4 disabled_cpus=0 cpus=1
(XEN) x86_max_cores=1 x86_num_siblings=1
(XEN) first=0
(XEN) weight=4
(XEN) cpu 0
(XEN) cpu 2
(XEN) cpu 4
(XEN) cpu 6
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
1-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #1 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #1......
(XEN) .... register #00: 01000000
(XEN) ....... : physical APIC id: 01
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 01000000
(XEN) ....... : arbitration: 01
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9214 MHz.
(XEN) ..... host bus clock speed is 92.5796 MHz.
(XEN) ..... bus_scale = 0x5ece
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x2
(XEN) detect_extended_topology initial_apicid=2 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x2
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=2
socket_cpumask[socket]=0000000000000000
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x4
(XEN) detect_extended_topology initial_apicid=4 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x4
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=4
socket_cpumask[socket]=0000000000000000
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) masked ExtINT on CPU#3
(XEN) smp_store_cpu_info id=3
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x6
(XEN) detect_extended_topology initial_apicid=6 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x6
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=3 socket=6
socket_cpumask[socket]=0000000000000000
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 4 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 600 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
(XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
(XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
(XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
(XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
(XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
(XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
(XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xffffffff80000000
(XEN) elf_paddr_offset = 0x0
(XEN) virt_offset = 0xffffffff80000000
(XEN) virt_kstart = 0xffffffff81000000
(XEN) virt_kend = 0xffffffff83000000
(XEN) virt_entry = 0xffffffff81f991f0
(XEN) p2m_base = 0xffffffffffffffff
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
be allocated)
(XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
(XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
(XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
(XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
(XEN) Page tables: ffffffff8412d000->ffffffff84152000
(XEN) Boot stack: ffffffff84152000->ffffffff84153000
(XEN) TOTAL: ffffffff80000000->ffffffff84400000
(XEN) ENTRY ADDRESS: ffffffff81f991f0
(XEN) Dom0 has maximum 4 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
(XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
(XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
(XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
(XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
(XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
(XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
(XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 304kB init memory.
mapping kernel into physical memory
about to get started...
If cpuid 0xb returned 1 rather than 0 in eax[4:0], we would get
consecutively-numbered physical processor IDs.
But the only requirement I see in the IA SDM (vol 2A, table 3-17) is that
the eax[4:0] value yield unique IDs, not necessarily consecutive. Likewise
while the examples in vol 3A sec 8.9 show physical IDs numbered
consecutively, the algorithms do not assume this is the case.
--Ed
On Mon, Nov 23, 2015 at 2:18 AM, Jan Beulich <jbeul...@suse.com> wrote:
> >>> On 21.11.15 at 02:21, <eswi...@skyportsystems.com> wrote:
> > The problem is that the index of the socket_cpumask array is derived via
> > cpu_to_socket() from the APIC ID of the processor in a given socket, but
> > the size of the array is computed based on nr_sockets, which is not
> > necessarily equal to the maximum APIC ID.
> >
> > Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems
> safer,
> > though a bit wasteful. I verified that this change fixes the boot crash
> > with 4 or 8 CPUs on VMware Fusion.
>
> But that raises the question of sanity of the CPUID output Xen gets
> presented: With
>
> > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> > (XEN) Processor #2 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> > (XEN) Processor #4 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> > (XEN) Processor #6 6:6 APIC version 21
>
> and taking the output you added, I can only suspect that the value
> used for determining the socket shift is unexpected (CPUID leaf 0xb).
> Could you supply the observed values? (See
> detect_extended_topology() and set_nr_sockets().) As you can
> see, the core IDs ("CPU: Physical Processor ID: ...") aren't sequential,
> which we expect them to be (with holes left only when non-power-of-2
> values need taking care of).
>
> Jan
>
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
