Bug#491309: NR_CPUS in debian linux-images
Bastian Blank wrote: > On Tue, Jul 29, 2008 at 07:48:11AM -0700, Mike Travis wrote: >> You could look at the 2.6.27 version of include/asm-x86/irq_vectors.h to see >> how NR_IRQ's is somewhat lessened (at least from a NR_CPUS=4096 perspective.) > > May this change be applicable to 2.6.26 or will it break something else? > > Bastian > I believe it is ok. The number of irq's is not really based on NR_CPUS but the number of IO_APICS. But I can't say for sure as I've been solely focused on 2.6.27 and haven't been following the breakup of NR_IRQS, NR_IRQ_VECTORS and NR_VECTORS and how they are used in various places. Btw, the biggest problem I found with increasing NR_CPUS was overflowing the stack, though I doubt increasing to 96 or 128 will trigger any of the hot spots fixed in 2.6.27. Thanks, Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#491309: NR_CPUS in debian linux-images
On Tue, Jul 29, 2008 at 05:19:22PM +0200, Bastian Blank wrote:
> On Tue, Jul 29, 2008 at 07:48:11AM -0700, Mike Travis wrote:
> > You could look at the 2.6.27 version of include/asm-x86/irq_vectors.h to see
> > how NR_IRQ's is somewhat lessened (at least from a NR_CPUS=4096
> > perspective.)
> May this change be applicable to 2.6.26 or will it break something else?
Aka something like:
diff --git a/include/asm-x86/irq_64.h b/include/asm-x86/irq_64.h
index 083d35a..30e1a6b 100644
--- a/include/asm-x86/irq_64.h
+++ b/include/asm-x86/irq_64.h
@@ -31,8 +31,22 @@
#define FIRST_SYSTEM_VECTOR0xef /* duplicated in hw_irq.h */
-#define NR_IRQS (NR_VECTORS + (32 * NR_CPUS))
-#define NR_IRQ_VECTORS NR_IRQS
+#if defined(CONFIG_X86_IO_APIC) || defined(CONFIG_PARAVIRT) ||
defined(CONFIG_X86_VISWS)
+
+# define NR_IRQS 224
+
+# if (224 >= 32 * NR_CPUS)
+# define NR_IRQ_VECTORS NR_IRQS
+# else
+# define NR_IRQ_VECTORS (32 * NR_CPUS)
+# endif
+
+#else /* IO_APIC || PARAVIRT */
+
+# define NR_IRQS 16
+# define NR_IRQ_VECTORSNR_IRQS
+
+#endif
static inline int irq_canonicalize(int irq)
{
Which produces (with 255 cpus):
textdata bss dec hex filename
3532199 505256 610456 4647911 46ebe7 vmlinux
Bastian
--
It would be illogical to kill without reason.
-- Spock, "Journey to Babel", stardate 3842.4
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#491309: NR_CPUS in debian linux-images
On Tue, Jul 29, 2008 at 07:48:11AM -0700, Mike Travis wrote: > You could look at the 2.6.27 version of include/asm-x86/irq_vectors.h to see > how NR_IRQ's is somewhat lessened (at least from a NR_CPUS=4096 perspective.) May this change be applicable to 2.6.26 or will it break something else? Bastian -- I'm a soldier, not a diplomat. I can only tell the truth. -- Kirk, "Errand of Mercy", stardate 3198.9 signature.asc Description: Digital signature
Bug#491309: NR_CPUS in debian linux-images
maximilian attems wrote:
> hello
>
> after getting an irc query and bug report on our currently set number of
> cpu for the upcoming lenny release:
> /boot/config-2.6.26-1-amd64:CONFIG_NR_CPUS=32
>
> -> http://bugs.debian.org/491309
>
> we came to this issue:
> 10:13 1345623 3148364 417112 4911099 4aeffb x86_64-255/vmlinux
> 10:13 1339887 380556 273880 1994323 1e6e53 x86_64-32/vmlinux
> 10:13 only change is the number of cpus *gna*
>
> 10:16 struct irq_cfg irq_cfg[NR_IRQS] __read_mostly = {
> 10:18 yeah, 2.5MiB just for the interrups
> 10:19 #define NR_IRQS (256 + (32 * NR_CPUS))
> 10:29 and because some parts have static initializers, it can't be
> moved
> into the bss section
>
> the debian installer cares about the image size.
>
> thanks for your input, as i see that you do the 4k cpu work on 2.6.27.
>
Hi,
Yes, we are aiming at having NR_CPUS=4096 as the default for distros using
2.6.27. The above structure is by far the biggest memory hog (along with the
companion irq_desc[]) and we've targeted the next release to go to dynamically
allocated irq's (which btw, hopefully will have NR_CPUS=16384.) The problem
with irq_cfg is that each element contains a cpumask_t field so growing NR_CPUS
causes an exponential increase.
You could look at the 2.6.27 version of include/asm-x86/irq_vectors.h to see
how NR_IRQ's is somewhat lessened (at least from a NR_CPUS=4096 perspective.)
Thanks,
Mike
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: NR_CPUS in debian linux-images
Hi,
from cujrrent intel and amd roadmaps, expect 4-way and 8-way 12 core
boxes during the lenny life cycle.
I think NR_CPUS should really be set to 96, at least.
On Tue, Jul 29, 2008 at 10:25:32AM +0200, maximilian attems wrote:
> we came to this issue:
> 10:13 1345623 3148364 417112 4911099 4aeffb x86_64-255/vmlinux
> 10:13 1339887 380556 273880 1994323 1e6e53 x86_64-32/vmlinux
> 10:13 only change is the number of cpus *gna*
>
> 10:16 struct irq_cfg irq_cfg[NR_IRQS] __read_mostly = {
> 10:18 yeah, 2.5MiB just for the interrups
> 10:19 #define NR_IRQS (256 + (32 * NR_CPUS))
> 10:29 and because some parts have static initializers, it can't be
> moved
> into the bss section
>
> the debian installer cares about the image size.
What is the actual limit?
Not raising this setting means loosing even more on the enterprise and
HPC terrain.
Best regards
Frederik Schüler
--
ENOSIG
signature.asc
Description: Digital signature
NR_CPUS in debian linux-images
hello
after getting an irc query and bug report on our currently set number of
cpu for the upcoming lenny release:
/boot/config-2.6.26-1-amd64:CONFIG_NR_CPUS=32
-> http://bugs.debian.org/491309
we came to this issue:
10:13 1345623 3148364 417112 4911099 4aeffb x86_64-255/vmlinux
10:13 1339887 380556 273880 1994323 1e6e53 x86_64-32/vmlinux
10:13 only change is the number of cpus *gna*
10:16 struct irq_cfg irq_cfg[NR_IRQS] __read_mostly = {
10:18 yeah, 2.5MiB just for the interrups
10:19 #define NR_IRQS (256 + (32 * NR_CPUS))
10:29 and because some parts have static initializers, it can't be moved
into the bss section
the debian installer cares about the image size.
thanks for your input, as i see that you do the 4k cpu work on 2.6.27.
--
maks
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
