subject:"How many CPU cores does FreeBSD support\?"

Re: How many CPU cores does FreeBSD support?

2017-01-06 Thread John Baldwin

On Wednesday, January 04, 2017 09:59:09 PM Konstantin Belousov wrote:
> On Wed, Jan 04, 2017 at 06:53:23PM +, Eric Joyner wrote:
> > Adding freebsd-current, because that's a good idea.
> > 
> > I see these lines in the beginning of dmesg:
> > 
> > MADT: Ignoring local APIC ID 256 (too high)
> > 
> > 
> >   [907/911]
> > MADT: Ignoring local APIC ID 260 (too high)
> > 
> > 
> >   [906/911]
> > MADT: Ignoring local APIC ID 264 (too high)
> > 
> > 
> >   [905/911]
> > MADT: Ignoring local APIC ID 268 (too high)
> > 
> > 
> >   [904/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> > 
> > 
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> > 
> > 
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> > 
> > 
> >   [901/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> > 
> > 
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> > 
> > 
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> > 
> > 
> >   [901/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> > 
> > 
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> > MADT: Ignoring local APIC ID 284 (too high)
> > MADT: Ignoring local APIC ID 288 (too high)
> > MADT: Ignoring local APIC ID 292 (too high)
> > MADT: Ignoring local APIC ID 296 (too high)
> > MADT: Ignoring local APIC ID 300 (too high)
> > MADT: Ignoring local APIC ID 257 (too high)
> > MADT: Ignoring local APIC ID 261 (too high)
> > MADT: Ignoring local APIC ID 265 (too high)
> > MADT: Ignoring local APIC ID 269 (too high)
> > MADT: Ignoring local APIC ID 273 (too high)
> > MADT: Ignoring local APIC ID 277 (too high)
> > MADT: Ignoring local APIC ID 281 (too high)
> > MADT: Ignoring local APIC ID 285 (too high)
> > MADT: Ignoring local APIC ID 289 (too high)
> > MADT: Ignoring local APIC ID 293 (too high)
> > MADT: Ignoring local APIC ID 297 (too high)
> > MADT: Ignoring local APIC ID 301 (too high)
> > MADT: Ignoring local APIC ID 258 (too high)
> > MADT: Ignoring local APIC ID 262 (too high)
> > MADT: Ignoring local APIC ID 266 (too high)
> > MADT: Ignoring local APIC ID 270 (too high)
> > MADT: Ignoring local APIC ID 274 (too high)
> > MADT: Ignoring local APIC ID 278 (too high)
> > MADT: Ignoring local APIC ID 282 (too high)
> > MADT: Ignoring local APIC ID 286 (too high)
> > MADT: Ignoring local APIC ID 290 (too high)
> > MADT: Ignoring local APIC ID 294 (too high)
> > MADT: Ignoring local APIC ID 298 (too high)
> > MADT: Ignoring local APIC ID 302 (too high)
> > MADT: Ignoring local APIC ID 255 (too high)
> > MADT: Ignoring local APIC ID 259 (too high)
> > MADT: Ignoring local APIC ID 263 (too high)
> > MADT: Ignoring local APIC ID 267 (too high)
> > MADT: Ignoring local APIC ID 271 (too high)
> > MADT: Ignoring local APIC ID 275 (too high)
> > MADT: Ignoring local APIC ID 279 (too high)
> > MADT: Ignoring local APIC ID 283 (too high)
> > MADT: Ignoring local APIC ID 287 (too high)
> > MADT: Ignoring local APIC ID 291 (too high)
> > MADT: Ignoring local APIC ID 295 (too high)
> > MADT: Ignoring local APIC ID 299 (too high)
> > MADT: Ignoring local APIC ID 303 (too high)
> > Copyright (c) 1992-2016 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> > The Regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
> > r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
> > 3.8.0)
> > SRAT: No CPU found for memory domain 1
> > VT(efifb): resolution 800x600
> > CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
> >   Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1
> > 
> > Features=0xbfebfbff
> > 
> > Features2=0x7ff8f39f
> >   AMD Features=0x2c100800
> >   AMD Features2=0x121
> >   Structured Extended
> > Features=0x1c0d23ab
> >   Structured Extended Features2=0x1
> >   XSAVE Features=0x1
> >   TSC: P-state invariant, performance statistics
> > real memory  = 223332007936 (212986 MB)
> > avail memory = 216976429056 (206924 MB)
> > Event timer "LAPIC" quality 600
> > ACPI APIC Table: 
> > FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs
> > FreeBSD/SMP: Non-uniform topology
> > 
> > It sounds like those MADT errors point to the problem?
> 
> In sys/x86/acpica/madt.c file, function madt_add_cpu(), just remove
> the block
>   if

Re: How many CPU cores does FreeBSD support?

2017-01-05 Thread Konstantin Belousov

On Thu, Jan 05, 2017 at 01:23:55AM +, Eric Joyner wrote:
> I started off with a snapshot of 12-CURRENT, but I got a kernel panic on
> boot that complained about a duplicate local APIC ID. For now, disabling
> Hyper Threading gets rid of that panic. I think there's a KASSERT that
> wasn't getting hit on my 11.0-RELEASE install.
This is because 11.0 has INVARIANTS disabled in the GENERIC kernel config.
Do you mean this one
KASSERT(la->la_enabled == 0, ("Duplicate local APIC ID %u", apic_id));
?  This is most likely to out of bounds accesses to the array.

Please revert the modification I suggested in previous mail, i.e. return
to the stock HEAD, and use this one instead.  You would also need to
bump MAXCPU in your kernel config.

If this works out, the patch still requires more work to get rid of that 
dozen of statically sized arrays.  But more likely, the patch requires
some debugging.

diff --git a/sys/x86/acpica/madt.c b/sys/x86/acpica/madt.c
index 959530c0d89..4509bc4a2f2 100644
--- a/sys/x86/acpica/madt.c
+++ b/sys/x86/acpica/madt.c
@@ -135,7 +135,7 @@ madt_setup_local(void)
vm_paddr_t dmartbl_physaddr;
const char *reason;
char *hw_vendor;
-   u_int p[4];
+   u_int p[4], i;
int user_x2apic;
bool bios_x2apic;
 
@@ -216,6 +216,11 @@ madt_setup_local(void)
(int)sizeof(madt->Header.OemId), madt->Header.OemId,
(int)sizeof(madt->Header.OemTableId), madt->Header.OemTableId);
 
+   if (!x2apic_mode) {
+   for (i = xAPIC_MAX_APIC_ID; i < MAX_APIC_ID; i++)
+   lapic_ignore(i);
+   }
+
/*
 * We ignore 64-bit local APIC override entries.  Should we
 * perhaps emit a warning here if we find one?
@@ -242,7 +247,7 @@ madt_setup_io(void)
panic("Using MADT but ACPI doesn't work");
}
 
-   ioapics = malloc(sizeof(*ioapics) * (MAX_APIC_ID + 1), M_MADT,
+   ioapics = malloc(sizeof(*ioapics) * (xAPIC_MAX_APIC_ID + 1), M_MADT,
M_WAITOK | M_ZERO);
 
/* First, we run through adding I/O APIC's. */
@@ -269,7 +274,7 @@ madt_setup_io(void)
}
 
/* Third, we register all the I/O APIC's. */
-   for (i = 0; i <= MAX_APIC_ID; i++)
+   for (i = 0; i <= xAPIC_MAX_APIC_ID; i++)
if (ioapics[i].io_apic != NULL)
ioapic_register(ioapics[i].io_apic);
 
@@ -365,7 +370,7 @@ madt_parse_apics(ACPI_SUBTABLE_HEADER *entry, void *arg 
__unused)
"MADT: Found IO APIC ID %u, Interrupt %u at %p\n",
apic->Id, apic->GlobalIrqBase,
(void *)(uintptr_t)apic->Address);
-   if (apic->Id > MAX_APIC_ID)
+   if (apic->Id > xAPIC_MAX_APIC_ID)
panic("%s: I/O APIC ID %u too high", __func__,
apic->Id);
if (ioapics[apic->Id].io_apic != NULL)
diff --git a/sys/x86/include/apicvar.h b/sys/x86/include/apicvar.h
index 09c3a638df8..b65c4a8c913 100644
--- a/sys/x86/include/apicvar.h
+++ b/sys/x86/include/apicvar.h
@@ -74,8 +74,9 @@
  * I/O device!
  */
 
-#defineMAX_APIC_ID 0xfe
-#defineAPIC_ID_ALL 0xff
+#definexAPIC_MAX_APIC_ID   0xfe
+#defineMAX_APIC_ID 0x200
+#defineAPIC_ID_ALL 0x
 
 /* I/O Interrupts are used for external devices such as ISA, PCI, etc. */
 #defineAPIC_IO_INTS(IDT_IO_INTS + 16)
@@ -204,6 +205,7 @@ int ioapic_set_smi(void *cookie, u_int pin);
  */
 struct apic_ops {
void(*create)(u_int, int);
+   void(*ignore)(u_int);
void(*init)(vm_paddr_t);
void(*xapic_mode)(void);
bool(*is_x2apic)(void);
@@ -256,6 +258,13 @@ lapic_create(u_int apic_id, int boot_cpu)
 }
 
 static inline void
+lapic_ignore(u_int apic_id)
+{
+
+   apic_ops.ignore(apic_id);
+}
+
+static inline void
 lapic_init(vm_paddr_t addr)
 {
 
diff --git a/sys/x86/include/x86_smp.h b/sys/x86/include/x86_smp.h
index 84a0eba25bc..0cf4ef234e4 100644
--- a/sys/x86/include/x86_smp.h
+++ b/sys/x86/include/x86_smp.h
@@ -78,6 +78,7 @@ inthand_t
 /* functions in x86_mp.c */
 void   assign_cpu_ids(void);
 void   cpu_add(u_int apic_id, char boot_cpu);
+void   cpu_ignore(u_int apic_id);
 void   cpustop_handler(void);
 void   cpususpend_handler(void);
 void   init_secondary_tail(void);
diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
index 1b1547de780..163f5e4b66f 100644
--- a/sys/x86/x86/local_apic.c
+++ b/sys/x86/x86/local_apic.c
@@ -298,6 +298,7 @@ struct pic lapic_pic = { .pic_resume = lapic_resume };
 
 /* Forward declarations for apic_ops */
 static voidnative_lapic_create(u_int apic_id, int boot_cpu);
+static voidnative_lapic_ignore(u_int apic_id);
 static voidnative_lapic_init(vm_paddr_t addr);
 static voidnative_lapic_xapic_mode(void);
 static voidnative_lapic_setup(int boot);
@@

Re: How many CPU cores does FreeBSD support?

2017-01-04 Thread Eric Joyner

I started off with a snapshot of 12-CURRENT, but I got a kernel panic on
boot that complained about a duplicate local APIC ID. For now, disabling
Hyper Threading gets rid of that panic. I think there's a KASSERT that
wasn't getting hit on my 11.0-RELEASE install.

On Wed, Jan 4, 2017 at 2:00 PM Eric Joyner  wrote:

> Ok; I will try that and will report back.
>
> - Eric
>
> On Wed, Jan 4, 2017 at 11:59 AM Konstantin Belousov 
> wrote:
>
> On Wed, Jan 04, 2017 at 06:53:23PM +, Eric Joyner wrote:
> > Adding freebsd-current, because that's a good idea.
> >
> > I see these lines in the beginning of dmesg:
> >
> > MADT: Ignoring local APIC ID 256 (too high)
> >
> >
> >   [907/911]
> > MADT: Ignoring local APIC ID 260 (too high)
> >
> >
> >   [906/911]
> > MADT: Ignoring local APIC ID 264 (too high)
> >
> >
> >   [905/911]
> > MADT: Ignoring local APIC ID 268 (too high)
> >
> >
> >   [904/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> >
> >
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> >
> >
> >   [901/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> >
> >
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> >
> >
> >   [901/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> > MADT: Ignoring local APIC ID 284 (too high)
> > MADT: Ignoring local APIC ID 288 (too high)
> > MADT: Ignoring local APIC ID 292 (too high)
> > MADT: Ignoring local APIC ID 296 (too high)
> > MADT: Ignoring local APIC ID 300 (too high)
> > MADT: Ignoring local APIC ID 257 (too high)
> > MADT: Ignoring local APIC ID 261 (too high)
> > MADT: Ignoring local APIC ID 265 (too high)
> > MADT: Ignoring local APIC ID 269 (too high)
> > MADT: Ignoring local APIC ID 273 (too high)
> > MADT: Ignoring local APIC ID 277 (too high)
> > MADT: Ignoring local APIC ID 281 (too high)
> > MADT: Ignoring local APIC ID 285 (too high)
> > MADT: Ignoring local APIC ID 289 (too high)
> > MADT: Ignoring local APIC ID 293 (too high)
> > MADT: Ignoring local APIC ID 297 (too high)
> > MADT: Ignoring local APIC ID 301 (too high)
> > MADT: Ignoring local APIC ID 258 (too high)
> > MADT: Ignoring local APIC ID 262 (too high)
> > MADT: Ignoring local APIC ID 266 (too high)
> > MADT: Ignoring local APIC ID 270 (too high)
> > MADT: Ignoring local APIC ID 274 (too high)
> > MADT: Ignoring local APIC ID 278 (too high)
> > MADT: Ignoring local APIC ID 282 (too high)
> > MADT: Ignoring local APIC ID 286 (too high)
> > MADT: Ignoring local APIC ID 290 (too high)
> > MADT: Ignoring local APIC ID 294 (too high)
> > MADT: Ignoring local APIC ID 298 (too high)
> > MADT: Ignoring local APIC ID 302 (too high)
> > MADT: Ignoring local APIC ID 255 (too high)
> > MADT: Ignoring local APIC ID 259 (too high)
> > MADT: Ignoring local APIC ID 263 (too high)
> > MADT: Ignoring local APIC ID 267 (too high)
> > MADT: Ignoring local APIC ID 271 (too high)
> > MADT: Ignoring local APIC ID 275 (too high)
> > MADT: Ignoring local APIC ID 279 (too high)
> > MADT: Ignoring local APIC ID 283 (too high)
> > MADT: Ignoring local APIC ID 287 (too high)
> > MADT: Ignoring local APIC ID 291 (too high)
> > MADT: Ignoring local APIC ID 295 (too high)
> > MADT: Ignoring local APIC ID 299 (too high)
> > MADT: Ignoring local APIC ID 303 (too high)
> > Copyright (c) 1992-2016 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> > The Regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
> > r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on
> LLVM
> > 3.8.0)
> > SRAT: No CPU found for memory domain 1
> > VT(efifb): resolution 800x600
> > CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
> >   Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1
> >
> >
> Features=0xbfebfbff
> >
> >
> Features2=0x7ff8f39f
> >   AMD Features=0x2c100800
> >   AMD Features2=0x121
> >   Structured Extended
> >
> Features=0x1c0d23ab
> >   Structured Extended Features2=0x1
> >   XSAVE Features=0x1
> >   TSC: P-state invariant, performance statistics
> > real

Re: How many CPU cores does FreeBSD support?

2017-01-04 Thread Eric Joyner

Ok; I will try that and will report back.

- Eric

On Wed, Jan 4, 2017 at 11:59 AM Konstantin Belousov 
wrote:

> On Wed, Jan 04, 2017 at 06:53:23PM +, Eric Joyner wrote:
> > Adding freebsd-current, because that's a good idea.
> >
> > I see these lines in the beginning of dmesg:
> >
> > MADT: Ignoring local APIC ID 256 (too high)
> >
> >
> >   [907/911]
> > MADT: Ignoring local APIC ID 260 (too high)
> >
> >
> >   [906/911]
> > MADT: Ignoring local APIC ID 264 (too high)
> >
> >
> >   [905/911]
> > MADT: Ignoring local APIC ID 268 (too high)
> >
> >
> >   [904/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> >
> >
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> >
> >
> >   [901/911]
> > MADT: Ignoring local APIC ID 272 (too high)
> >
> >
> >   [903/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> >
> >
> >   [901/911]
> > MADT: Ignoring local APIC ID 276 (too high)
> >
> >
> >   [902/911]
> > MADT: Ignoring local APIC ID 280 (too high)
> > MADT: Ignoring local APIC ID 284 (too high)
> > MADT: Ignoring local APIC ID 288 (too high)
> > MADT: Ignoring local APIC ID 292 (too high)
> > MADT: Ignoring local APIC ID 296 (too high)
> > MADT: Ignoring local APIC ID 300 (too high)
> > MADT: Ignoring local APIC ID 257 (too high)
> > MADT: Ignoring local APIC ID 261 (too high)
> > MADT: Ignoring local APIC ID 265 (too high)
> > MADT: Ignoring local APIC ID 269 (too high)
> > MADT: Ignoring local APIC ID 273 (too high)
> > MADT: Ignoring local APIC ID 277 (too high)
> > MADT: Ignoring local APIC ID 281 (too high)
> > MADT: Ignoring local APIC ID 285 (too high)
> > MADT: Ignoring local APIC ID 289 (too high)
> > MADT: Ignoring local APIC ID 293 (too high)
> > MADT: Ignoring local APIC ID 297 (too high)
> > MADT: Ignoring local APIC ID 301 (too high)
> > MADT: Ignoring local APIC ID 258 (too high)
> > MADT: Ignoring local APIC ID 262 (too high)
> > MADT: Ignoring local APIC ID 266 (too high)
> > MADT: Ignoring local APIC ID 270 (too high)
> > MADT: Ignoring local APIC ID 274 (too high)
> > MADT: Ignoring local APIC ID 278 (too high)
> > MADT: Ignoring local APIC ID 282 (too high)
> > MADT: Ignoring local APIC ID 286 (too high)
> > MADT: Ignoring local APIC ID 290 (too high)
> > MADT: Ignoring local APIC ID 294 (too high)
> > MADT: Ignoring local APIC ID 298 (too high)
> > MADT: Ignoring local APIC ID 302 (too high)
> > MADT: Ignoring local APIC ID 255 (too high)
> > MADT: Ignoring local APIC ID 259 (too high)
> > MADT: Ignoring local APIC ID 263 (too high)
> > MADT: Ignoring local APIC ID 267 (too high)
> > MADT: Ignoring local APIC ID 271 (too high)
> > MADT: Ignoring local APIC ID 275 (too high)
> > MADT: Ignoring local APIC ID 279 (too high)
> > MADT: Ignoring local APIC ID 283 (too high)
> > MADT: Ignoring local APIC ID 287 (too high)
> > MADT: Ignoring local APIC ID 291 (too high)
> > MADT: Ignoring local APIC ID 295 (too high)
> > MADT: Ignoring local APIC ID 299 (too high)
> > MADT: Ignoring local APIC ID 303 (too high)
> > Copyright (c) 1992-2016 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> > The Regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
> > r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on
> LLVM
> > 3.8.0)
> > SRAT: No CPU found for memory domain 1
> > VT(efifb): resolution 800x600
> > CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
> >   Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1
> >
> >
> Features=0xbfebfbff
> >
> >
> Features2=0x7ff8f39f
> >   AMD Features=0x2c100800
> >   AMD Features2=0x121
> >   Structured Extended
> >
> Features=0x1c0d23ab
> >   Structured Extended Features2=0x1
> >   XSAVE Features=0x1
> >   TSC: P-state invariant, performance statistics
> > real memory  = 223332007936 (212986 MB)
> > avail memory = 216976429056 (206924 MB)
> > Event timer "LAPIC" quality 600
> > ACPI APIC Table: 
> > FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs
> > FreeBSD/SMP: Non-uniform topology
> >
> > It sounds like those MADT errors point to the problem?
>
> In sys/x86/acpica/madt.c file, function

Re: How many CPU cores does FreeBSD support?

2017-01-04 Thread Konstantin Belousov

On Wed, Jan 04, 2017 at 06:53:23PM +, Eric Joyner wrote:
> Adding freebsd-current, because that's a good idea.
> 
> I see these lines in the beginning of dmesg:
> 
> MADT: Ignoring local APIC ID 256 (too high)
> 
> 
>   [907/911]
> MADT: Ignoring local APIC ID 260 (too high)
> 
> 
>   [906/911]
> MADT: Ignoring local APIC ID 264 (too high)
> 
> 
>   [905/911]
> MADT: Ignoring local APIC ID 268 (too high)
> 
> 
>   [904/911]
> MADT: Ignoring local APIC ID 272 (too high)
> 
> 
>   [903/911]
> MADT: Ignoring local APIC ID 276 (too high)
> 
> 
>   [902/911]
> MADT: Ignoring local APIC ID 280 (too high)
> 
> 
>   [901/911]
> MADT: Ignoring local APIC ID 272 (too high)
> 
> 
>   [903/911]
> MADT: Ignoring local APIC ID 276 (too high)
> 
> 
>   [902/911]
> MADT: Ignoring local APIC ID 280 (too high)
> 
> 
>   [901/911]
> MADT: Ignoring local APIC ID 276 (too high)
> 
> 
>   [902/911]
> MADT: Ignoring local APIC ID 280 (too high)
> MADT: Ignoring local APIC ID 284 (too high)
> MADT: Ignoring local APIC ID 288 (too high)
> MADT: Ignoring local APIC ID 292 (too high)
> MADT: Ignoring local APIC ID 296 (too high)
> MADT: Ignoring local APIC ID 300 (too high)
> MADT: Ignoring local APIC ID 257 (too high)
> MADT: Ignoring local APIC ID 261 (too high)
> MADT: Ignoring local APIC ID 265 (too high)
> MADT: Ignoring local APIC ID 269 (too high)
> MADT: Ignoring local APIC ID 273 (too high)
> MADT: Ignoring local APIC ID 277 (too high)
> MADT: Ignoring local APIC ID 281 (too high)
> MADT: Ignoring local APIC ID 285 (too high)
> MADT: Ignoring local APIC ID 289 (too high)
> MADT: Ignoring local APIC ID 293 (too high)
> MADT: Ignoring local APIC ID 297 (too high)
> MADT: Ignoring local APIC ID 301 (too high)
> MADT: Ignoring local APIC ID 258 (too high)
> MADT: Ignoring local APIC ID 262 (too high)
> MADT: Ignoring local APIC ID 266 (too high)
> MADT: Ignoring local APIC ID 270 (too high)
> MADT: Ignoring local APIC ID 274 (too high)
> MADT: Ignoring local APIC ID 278 (too high)
> MADT: Ignoring local APIC ID 282 (too high)
> MADT: Ignoring local APIC ID 286 (too high)
> MADT: Ignoring local APIC ID 290 (too high)
> MADT: Ignoring local APIC ID 294 (too high)
> MADT: Ignoring local APIC ID 298 (too high)
> MADT: Ignoring local APIC ID 302 (too high)
> MADT: Ignoring local APIC ID 255 (too high)
> MADT: Ignoring local APIC ID 259 (too high)
> MADT: Ignoring local APIC ID 263 (too high)
> MADT: Ignoring local APIC ID 267 (too high)
> MADT: Ignoring local APIC ID 271 (too high)
> MADT: Ignoring local APIC ID 275 (too high)
> MADT: Ignoring local APIC ID 279 (too high)
> MADT: Ignoring local APIC ID 283 (too high)
> MADT: Ignoring local APIC ID 287 (too high)
> MADT: Ignoring local APIC ID 291 (too high)
> MADT: Ignoring local APIC ID 295 (too high)
> MADT: Ignoring local APIC ID 299 (too high)
> MADT: Ignoring local APIC ID 303 (too high)
> Copyright (c) 1992-2016 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
> r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
> 3.8.0)
> SRAT: No CPU found for memory domain 1
> VT(efifb): resolution 800x600
> CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
>   Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1
> 
> Features=0xbfebfbff
> 
> Features2=0x7ff8f39f
>   AMD Features=0x2c100800
>   AMD Features2=0x121
>   Structured Extended
> Features=0x1c0d23ab
>   Structured Extended Features2=0x1
>   XSAVE Features=0x1
>   TSC: P-state invariant, performance statistics
> real memory  = 223332007936 (212986 MB)
> avail memory = 216976429056 (206924 MB)
> Event timer "LAPIC" quality 600
> ACPI APIC Table: 
> FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs
> FreeBSD/SMP: Non-uniform topology
> 
> It sounds like those MADT errors point to the problem?

In sys/x86/acpica/madt.c file, function madt_add_cpu(), just remove
the block
if (apic_id > MAX_APIC_ID) {
printf("MADT: Ignoring local APIC ID %u (too high)\n",
apic_id);
return;
}
Best to do it on HEAD and not on 11.0.
If it helps, commit the change.

Also, for >= 256 CPUs, you might need to enable interrupt remapping
in the DMAR, use loader

x2APIC support (Re: How many CPU cores does FreeBSD support?)

2017-01-04 Thread Ian FREISLICH


Hi

I think your issue relates to x2APIC support and I don't think that full 
support for parsing x2APIC MADT entries and doing interrupts remapping 
has been committed yet from my very brief search of the source and 
commit messages.  I've copied Andriy Gapon and Konstantin Belousov who 
participated in another x2APIC thread in September last year.  They will 
probably have a more intelligible response.


sys/x86/include/apicvar.h limits APIC IDs as follows:

#define MAX_APIC_ID 0xfe
#define APIC_ID_ALL 0xff

This is why CPUs with an APIC ID above 255 are being ignored in 
acpica/madt.c


sys/x86/x86/mptable.c has this choice piece of code:

if (intr->dst_apic_id == 0xff)
apic_id = APIC_ID_ALL;
else
apic_id = intr->dst_apic_id;

which for the current value of APIC_ID_ALL can simply be rewritten as:

apic_id = intr->dst_apic_id;

I do not think it is safe to increase MAX_APIC_ID yet.

Ian

--
Ian Freislich

On 01/04/17 13:53, Eric Joyner wrote:

Adding freebsd-current, because that's a good idea.

I see these lines in the beginning of dmesg:

MADT: Ignoring local APIC ID 256 (too high)


   [907/911]
MADT: Ignoring local APIC ID 260 (too high)


   [906/911]
MADT: Ignoring local APIC ID 264 (too high)


   [905/911]
MADT: Ignoring local APIC ID 268 (too high)


   [904/911]
MADT: Ignoring local APIC ID 272 (too high)


   [903/911]
MADT: Ignoring local APIC ID 276 (too high)


   [902/911]
MADT: Ignoring local APIC ID 280 (too high)


   [901/911]
MADT: Ignoring local APIC ID 272 (too high)


   [903/911]
MADT: Ignoring local APIC ID 276 (too high)


   [902/911]
MADT: Ignoring local APIC ID 280 (too high)


   [901/911]
MADT: Ignoring local APIC ID 276 (too high)


   [902/911]
MADT: Ignoring local APIC ID 280 (too high)
MADT: Ignoring local APIC ID 284 (too high)
MADT: Ignoring local APIC ID 288 (too high)
MADT: Ignoring local APIC ID 292 (too high)
MADT: Ignoring local APIC ID 296 (too high)
MADT: Ignoring local APIC ID 300 (too high)
MADT: Ignoring local APIC ID 257 (too high)
MADT: Ignoring local APIC ID 261 (too high)
MADT: Ignoring local APIC ID 265 (too high)
MADT: Ignoring local APIC ID 269 (too high)
MADT: Ignoring local APIC ID 273 (too high)
MADT: Ignoring local APIC ID 277 (too high)
MADT: Ignoring local APIC ID 281 (too high)
MADT: Ignoring local APIC ID 285 (too high)
MADT: Ignoring local APIC ID 289 (too high)
MADT: Ignoring local APIC ID 293 (too high)
MADT: Ignoring local APIC ID 297 (too high)
MADT: Ignoring local APIC ID 301 (too high)
MADT: Ignoring local APIC ID 258 (too high)
MADT: Ignoring local APIC ID 262 (too high)
MADT: Ignoring local APIC ID 266 (too high)
MADT: Ignoring local APIC ID 270 (too high)
MADT: Ignoring local APIC ID 274 (too high)
MADT: Ignoring local APIC ID 278 (too high)
MADT: Ignoring local APIC ID 282 (too high)
MADT: Ignoring local APIC ID 286 (too high)
MADT: Ignoring local APIC ID 290 (too high)
MADT: Ignoring local APIC ID 294 (too high)
MADT: Ignoring local APIC ID 298 (too high)
MADT: Ignoring local APIC ID 302 (too high)
MADT: Ignoring local APIC ID 255 (too high)
MADT: Ignoring local APIC ID 259 (too high)
MADT: Ignoring local APIC ID 263 (too high)
MADT: Ignoring local APIC ID 267 (too high)
MADT: Ignoring local APIC ID 271 (too high)
MADT: Ignoring local APIC ID 275 (too high)
MADT: Ignoring local APIC ID 279 (too high)
MADT: Ignoring local APIC ID 283 (too high)
MADT: Ignoring local APIC ID 287 (too high)
MADT: Ignoring local APIC ID 291 (too high)
MADT: Ignoring local APIC ID 295 (too high)
MADT: Ignoring local APIC ID 299 (too high)
MADT: Ignoring local APIC ID 303 (too high)
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
 r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
3.8.0)
SRAT: No CPU found for memory domain 1
VT(efifb): resolution 800x600
CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
   Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1

Features=0xbfebfbff

Features2=0x7ff8f39f
   AMD Features=0x2c100800
   AMD Features2=0x121
   Structured Extended
Features=0x1c0d23ab
   Structured Extended Features2=0x1
   XSAVE Features=0x1

Re: How many CPU cores does FreeBSD support?

2017-01-04 Thread Steve O'Hara-Smith

On Wed, 04 Jan 2017 18:53:23 +
Eric Joyner  wrote:

> MADT: Ignoring local APIC ID 303 (too high)

Hmm you said this machine has 272 cores so clearly the IDs have
gaps (I have no idea why there are gaps or where these IDs are allocated
though). I expect you'll find that 223 of the cores have IDs below 256 and
the remaining 49 have IDs of 256 and above and FreeBSD is not using those
cores.

-- 
Steve O'Hara-Smith 
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: How many CPU cores does FreeBSD support?

2017-01-04 Thread Eric Joyner

Adding freebsd-current, because that's a good idea.

I see these lines in the beginning of dmesg:

MADT: Ignoring local APIC ID 256 (too high)

  [907/911]
MADT: Ignoring local APIC ID 260 (too high)

  [906/911]
MADT: Ignoring local APIC ID 264 (too high)

  [905/911]
MADT: Ignoring local APIC ID 268 (too high)

  [904/911]
MADT: Ignoring local APIC ID 272 (too high)

  [903/911]
MADT: Ignoring local APIC ID 276 (too high)

  [902/911]
MADT: Ignoring local APIC ID 280 (too high)

  [901/911]
MADT: Ignoring local APIC ID 272 (too high)

  [903/911]
MADT: Ignoring local APIC ID 276 (too high)

  [902/911]
MADT: Ignoring local APIC ID 280 (too high)

  [901/911]
MADT: Ignoring local APIC ID 276 (too high)

  [902/911]
MADT: Ignoring local APIC ID 280 (too high)
MADT: Ignoring local APIC ID 284 (too high)
MADT: Ignoring local APIC ID 288 (too high)
MADT: Ignoring local APIC ID 292 (too high)
MADT: Ignoring local APIC ID 296 (too high)
MADT: Ignoring local APIC ID 300 (too high)
MADT: Ignoring local APIC ID 257 (too high)
MADT: Ignoring local APIC ID 261 (too high)
MADT: Ignoring local APIC ID 265 (too high)
MADT: Ignoring local APIC ID 269 (too high)
MADT: Ignoring local APIC ID 273 (too high)
MADT: Ignoring local APIC ID 277 (too high)
MADT: Ignoring local APIC ID 281 (too high)
MADT: Ignoring local APIC ID 285 (too high)
MADT: Ignoring local APIC ID 289 (too high)
MADT: Ignoring local APIC ID 293 (too high)
MADT: Ignoring local APIC ID 297 (too high)
MADT: Ignoring local APIC ID 301 (too high)
MADT: Ignoring local APIC ID 258 (too high)
MADT: Ignoring local APIC ID 262 (too high)
MADT: Ignoring local APIC ID 266 (too high)
MADT: Ignoring local APIC ID 270 (too high)
MADT: Ignoring local APIC ID 274 (too high)
MADT: Ignoring local APIC ID 278 (too high)
MADT: Ignoring local APIC ID 282 (too high)
MADT: Ignoring local APIC ID 286 (too high)
MADT: Ignoring local APIC ID 290 (too high)
MADT: Ignoring local APIC ID 294 (too high)
MADT: Ignoring local APIC ID 298 (too high)
MADT: Ignoring local APIC ID 302 (too high)
MADT: Ignoring local APIC ID 255 (too high)
MADT: Ignoring local APIC ID 259 (too high)
MADT: Ignoring local APIC ID 263 (too high)
MADT: Ignoring local APIC ID 267 (too high)
MADT: Ignoring local APIC ID 271 (too high)
MADT: Ignoring local APIC ID 275 (too high)
MADT: Ignoring local APIC ID 279 (too high)
MADT: Ignoring local APIC ID 283 (too high)
MADT: Ignoring local APIC ID 287 (too high)
MADT: Ignoring local APIC ID 291 (too high)
MADT: Ignoring local APIC ID 295 (too high)
MADT: Ignoring local APIC ID 299 (too high)
MADT: Ignoring local APIC ID 303 (too high)
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
3.8.0)
SRAT: No CPU found for memory domain 1
VT(efifb): resolution 800x600
CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x50671  Family=0x6  Model=0x57  Stepping=1

Features=0xbfebfbff

Features2=0x7ff8f39f
  AMD Features=0x2c100800
  AMD Features2=0x121
  Structured Extended
Features=0x1c0d23ab
  Structured Extended Features2=0x1
  XSAVE Features=0x1
  TSC: P-state invariant, performance statistics
real memory  = 223332007936 (212986 MB)
avail memory = 216976429056 (206924 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: 
FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs
FreeBSD/SMP: Non-uniform topology

It sounds like those MADT errors point to the problem?

- Eric

On Tue, Jan 3, 2017 at 11:41 PM Shane Ambler  wrote:

> On 04/01/2017 08:08, Eric Joyner wrote:
> > I have an 11.0-RELEASE-p1 system with:
> >
> > hw.model: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz
> >
> > It's got 68 cores, but with 4-way SMT, I think it should show up with 272
> > logical cores on FreeBSD.
> >
> > However, in both kern.sched.topology_spec and hw.ncpu, it shows there are
> > only 223 cores.
> >
> > Is the limit effectively 223 cores, even though I see some #define MAXCPU
> > 256 lines in the source?
>
> You will probably find freebsd-current a better mailing list for this.
>
> How many show in dmesg?
>
>  From https://svnweb.freebsd.org/base?view=revision=270223

Re: How many CPU cores does FreeBSD support?

Re: How many CPU cores does FreeBSD support?

Re: How many CPU cores does FreeBSD support?

Re: How many CPU cores does FreeBSD support?

Re: How many CPU cores does FreeBSD support?

x2APIC support (Re: How many CPU cores does FreeBSD support?)

Re: How many CPU cores does FreeBSD support?

Re: How many CPU cores does FreeBSD support?

8 matches

Site Navigation

Mail list logo

Footer information