Re: SMP on Debian server with Hyperthreading
Sincerely, - Original Message - From: Guus Houtzager [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, 06 September, 2003 6:07 PM Subject: Re: SMP on Debian server with Hyperthreading Hi, On Sat, 2003-09-06 at 09:31, Jason Lim wrote: Hi Guus, Yes, BIOS setting is enabled. The ONLY thing that I haven't done is edit lilo to include the acpismp=force setting. Did you set that to make it work? Does it work without it (ie. SMP enabled WITHOUT modifying lilo)? Haven't modified lilo for this to work. I looked again at your dmesg output and compared it to mine. With me it's ACPI doing the detection of the cpu's. Did you enable ACPI? Ah... did NOT have ACPI enabled. Did not know it had to be, for SMP to work! I enabled it, compiled the kernel, and /proc/cpuinfo sees 2 CPUs, but top doesn't. After spending ages trying to figure it out, I found the version of top that comes with Debian 3.0 does NOT have SMP support. So I downloaded the backported version from: http://people.debian.org/~nobse/debian/woody/backported/procps/ and after top launches, press 1 (or set the appropriate cmd line parameter) and it works great. I did ACPI a little different than you though... I have: # CONFIG_ACPI_HT_ONLY is not set set as y/1 (and therefore everything else set as n/0), as I don't need all the acpi_power and other stuff, just want SMP to work. So it's working now. Thanks for your help! Part of my dmesg: Linux version 2.4.22 ([EMAIL PROTECTED]) (gcc version 3.3.2 20030831 (Debian prerelease)) #1 SMP Tue Sep 2 11:37:18 CEST 2003 BIOS-provided physical RAM map: BIOS-e820: - 000a (usable) BIOS-e820: 000f - 0010 (reserved) BIOS-e820: 0010 - 2fff (usable) BIOS-e820: 2fff - 2fff3000 (ACPI NVS) BIOS-e820: 2fff3000 - 3000 (ACPI data) BIOS-e820: fec0 - 0001 (reserved) 767MB LOWMEM available. ACPI: have wakeup address 0xc0002000 found SMP MP-table at 000f51c0 hm, page 000f5000 reserved twice. hm, page 000f6000 reserved twice. hm, page 000f reserved twice. hm, page 000f1000 reserved twice. On node 0 totalpages: 196592 zone(0): 4096 pages. zone(1): 192496 pages. zone(2): 0 pages. ACPI: RSDP (v000 IntelR) @ 0x000f6c50 ACPI: RSDT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff3000 ACPI: FADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff3040 ACPI: MADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff6700 ACPI: DSDT (v001 INTELR AWRDACPI 0x1000 MSFT 0x010d) @ 0x ACPI: Local APIC address 0xfee0 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) Processor #0 Pentium 4(tm) XEON(tm) APIC version 16 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) Processor #1 Pentium 4(tm) XEON(tm) APIC version 16 ACPI: LAPIC_NMI (acpi_id[0x00] polarity[0x1] trigger[0x1] lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x00] polarity[0x1] trigger[0x1] lint[0x1]) ACPI: IOAPIC (id[0x02] address[0xfec0] global_irq_base[0x0]) IOAPIC[0]: Assigned apic_id 2 IOAPIC[0]: apic_id 2, version 32, address 0xfec0, IRQ 0-23 ACPI: INT_SRC_OVR (bus[0] irq[0x0] global_irq[0x2] polarity[0x0] trigger[0x0]) ACPI: INT_SRC_OVR (bus[0] irq[0x9] global_irq[0x9] polarity[0x1] trigger[0x3]) Using ACPI (MADT) for SMP configuration information Kernel command line: auto BOOT_IMAGE=Linux ro root=303 hdc=scsi and so on... ACPI part of my config: # # ACPI Support # CONFIG_ACPI=y # CONFIG_ACPI_HT_ONLY is not set CONFIG_ACPI_BOOT=y CONFIG_ACPI_BUS=y CONFIG_ACPI_INTERPRETER=y CONFIG_ACPI_EC=y CONFIG_ACPI_POWER=y CONFIG_ACPI_PCI=y CONFIG_ACPI_SLEEP=y CONFIG_ACPI_SYSTEM=y # CONFIG_ACPI_AC is not set # CONFIG_ACPI_BATTERY is not set CONFIG_ACPI_BUTTON=y CONFIG_ACPI_FAN=y CONFIG_ACPI_PROCESSOR=y # CONFIG_ACPI_THERMAL is not set # CONFIG_ACPI_ASUS is not set # CONFIG_ACPI_TOSHIBA is not set # CONFIG_ACPI_DEBUG is not set # CONFIG_ACPI_RELAXED_AML is not set Thanks. I hope this helps. -- Guus Houtzager Email: [EMAIL PROTECTED] PGP fingerprint = 5E E6 96 35 F0 64 34 14 CC 03 2B 36 71 FB 4B 5D A)bort, R)etry, I)nfluence with large hammer. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: SMP on Debian server with Hyperthreading
Hi, On Fri, 2003-09-05 at 17:06, Jason Lim wrote: Hi all, Just wondering... I've got a 2.4Ghz Hyperthreading (100% it is the hyperthreading model), and the BIOS sees it. I have a 3.06 GHz P4 on an I845G chipset and in the BIOS I had to switch the HT option from disabled to enabled. So is it actually on in your BIOS-settings? I then compiled the kernel... the usual, except added the SMP support setting Symmetric multi-processing support. Nothing else was changed. Compiled it, liloed it... it's running it: # uname -a Linux megalith 2.4.22 #8 SMP Thu Aug 28 14:44:13 HKT 2003 i686 unknown However, That's correct. # mpstat -P Not an SMP machine... And in top i don't see the multiple CPU usage This does work on my setup. I'm running 2.4.22 now, but it worked with 2.4.20 aswell. this is all strange. For Linux, aren't Hyperthreading CPUs suppose to act like completely separate independent CPUs (this was suppose to change in 2.6... but for 2.4, they can't tell the difference, right?). Hope you can advise... as hyperthreading is there but not being used, which is a waste and could add performance. My machine is a workstation, so I don't know what would happen with typical server software (ie apache, database) but for instance compiling a kernel with 3 simultaneous threads is about 20% faster than using only 1 thread. Thanks in advance! Hope this helps! Jas Regards, -- Guus Houtzager Email: [EMAIL PROTECTED] PGP fingerprint = 5E E6 96 35 F0 64 34 14 CC 03 2B 36 71 FB 4B 5D A)bort, R)etry, I)nfluence with large hammer. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: SMP on Debian server with Hyperthreading
On Fri, 2003-09-05 at 17:06, Jason Lim wrote: Hi all, snip Just wondering... I've got a 2.4Ghz Hyperthreading (100% it is the hyperthreading model), and the BIOS sees it. Hope you can advise... as hyperthreading is there but not being used, which is a waste and could add performance. Appended below is a mail from a freind of mine, his company does work with Open Source sytems for the larger enterprises in India. He came accross HT, and not finding any real life data, decided that the only task worth doing was a kernel compile. The report is good reading. I am cc:ing him, he is not subscribed to this list. Hope it helps. -- Sanjeev ==BEGIN== Intel has released this new feature in its high-end processors, where one processor can internally act as two processors in hardware. In essence, there are two sets of CPU registers, two caches, two TLBs, etc. But there's only one external address and data bus, so the interface between the CPU and the rest of the hardware is (largely) like that of a single processor. Look for Hyperthreading on www.intel.com. The interesting thing about hyperthreading is that it is done purely in hardware. This means that the OS kernel does not know the difference between two physical processors and an Intel Xeon with hyperthreading switched on. We recently got a client's machine for setting up a database server. We had asked for a two-processor machine, we got a dual-processor with Hyperthreading. You go to the ROM BIOS and switch on or switch off hyperthreading. If it is switched on, /proc/cpuinfo (we work in Linux) shows four processors. I wanted to see whether we get the power of four processors when we switch on hyperthreading, in a typical SMP Linux environment. I wanted to run a set of parallel Unix processes with and without hyperthreading, and see whether I got faster system throughput with four virtual processors than with two real ones. TYPE OF JOBS: I wanted jobs which would do some I/O but would do primarily a lot of in-memory data manipulation. And I didn't have the time to write custom code. So I chose C compilation. A make on a large source tree would give me a lot of this sort of workload. PARALLELISM CONTROL: I used make with the -j option of GNU Make. This controls how many parallel branches are fired by the top-level make for the compilations. It is clear that this is a less than perfect way to generate parallel workloads, because a full compilation of a complete source tree would not have a consistent degree of parallelism. I am certain that the last part of a compilation job would be sequential, but hopefully, with a large enough source tree with sufficiently large number of independent modules, 95%+ of the compilation would have opportunities for dozens of parallel threads. ACTUAL WORKLOAD: The final script that I ran would do the following actions, one after the other in /usr/src/linux: make -j $COUNT clean make -j $COUNT dep make -j $COUNT bzImage make -j $COUNT modules As you can see, this already shows you sequential points, when one make completes and the next make starts. The size of the job was quite huge. I ran make config first, and hit Enter and kept the key pressed. The resultant configuration has lots an lots of optional modules selected. For instance, the test compilations generate 2800+ .o files. The kernel which runs on my laptop generates just 650+ .o files when compiled. I could see that the workload was CPU-intensive; with high parallelism, I was getting CPU idle time less than 1%. And user-state CPU usage was 93%+, the rest being in system calls. This profile is expected based on the fact that there is large RAM availability for disk cacheing. MEASUREMENT METHOD: I ran the set of make commands and used the Bash $SECONDS variable to get the system clock, +/- 1 second. This error was okay; my kernel compilation took 2500+ seconds with zero parallelism. Moreover, with each parallelism setting, I ran the full set of make commands five times, taking the clocktime measurements each time. I averaged them using integer division. The error of +/- 5 seconds due to integer division again should not matter; typical job runtimes were always more than 1000 seconds. SYSTEM CONFIG: Two physical Intel Xeon processors at 1.8GHz (as per /proc/cpuinfo), 1 GB RAM, IDE drives, ext3 file system. (I also tried using ext2 filesystems, but got timings identical to ext3; the ext3 journalling does not seem to be adding any significant load.) OS kernel was Linux 2.4.19-64GB-SMP, a stock SuSE 8.1 SMP kernel. At peak loads, even with max parallelism, I never saw any swap space being used. This means that the only disk I/O must have been for writing out intermediate and output files to disk. I guess there was practically no page fault occurrence on the system during my test runs, though I didn't bother to verify this. Max RAM usage was about 970MB with -j 6 at certain points. The disks, though IDE, are fast. A cat /proc/ide/piix (that piix is
Re: SMP on Debian server with Hyperthreading
Hi, On Sat, 2003-09-06 at 09:31, Jason Lim wrote: Hi Guus, Yes, BIOS setting is enabled. The ONLY thing that I haven't done is edit lilo to include the acpismp=force setting. Did you set that to make it work? Does it work without it (ie. SMP enabled WITHOUT modifying lilo)? Haven't modified lilo for this to work. I looked again at your dmesg output and compared it to mine. With me it's ACPI doing the detection of the cpu's. Did you enable ACPI? Part of my dmesg: Linux version 2.4.22 ([EMAIL PROTECTED]) (gcc version 3.3.2 20030831 (Debian prerelease)) #1 SMP Tue Sep 2 11:37:18 CEST 2003 BIOS-provided physical RAM map: BIOS-e820: - 000a (usable) BIOS-e820: 000f - 0010 (reserved) BIOS-e820: 0010 - 2fff (usable) BIOS-e820: 2fff - 2fff3000 (ACPI NVS) BIOS-e820: 2fff3000 - 3000 (ACPI data) BIOS-e820: fec0 - 0001 (reserved) 767MB LOWMEM available. ACPI: have wakeup address 0xc0002000 found SMP MP-table at 000f51c0 hm, page 000f5000 reserved twice. hm, page 000f6000 reserved twice. hm, page 000f reserved twice. hm, page 000f1000 reserved twice. On node 0 totalpages: 196592 zone(0): 4096 pages. zone(1): 192496 pages. zone(2): 0 pages. ACPI: RSDP (v000 IntelR) @ 0x000f6c50 ACPI: RSDT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff3000 ACPI: FADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff3040 ACPI: MADT (v001 IntelR AWRDACPI 0x42302e31 AWRD 0x) @ 0x2fff6700 ACPI: DSDT (v001 INTELR AWRDACPI 0x1000 MSFT 0x010d) @ 0x ACPI: Local APIC address 0xfee0 ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) Processor #0 Pentium 4(tm) XEON(tm) APIC version 16 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled) Processor #1 Pentium 4(tm) XEON(tm) APIC version 16 ACPI: LAPIC_NMI (acpi_id[0x00] polarity[0x1] trigger[0x1] lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x00] polarity[0x1] trigger[0x1] lint[0x1]) ACPI: IOAPIC (id[0x02] address[0xfec0] global_irq_base[0x0]) IOAPIC[0]: Assigned apic_id 2 IOAPIC[0]: apic_id 2, version 32, address 0xfec0, IRQ 0-23 ACPI: INT_SRC_OVR (bus[0] irq[0x0] global_irq[0x2] polarity[0x0] trigger[0x0]) ACPI: INT_SRC_OVR (bus[0] irq[0x9] global_irq[0x9] polarity[0x1] trigger[0x3]) Using ACPI (MADT) for SMP configuration information Kernel command line: auto BOOT_IMAGE=Linux ro root=303 hdc=scsi and so on... ACPI part of my config: # # ACPI Support # CONFIG_ACPI=y # CONFIG_ACPI_HT_ONLY is not set CONFIG_ACPI_BOOT=y CONFIG_ACPI_BUS=y CONFIG_ACPI_INTERPRETER=y CONFIG_ACPI_EC=y CONFIG_ACPI_POWER=y CONFIG_ACPI_PCI=y CONFIG_ACPI_SLEEP=y CONFIG_ACPI_SYSTEM=y # CONFIG_ACPI_AC is not set # CONFIG_ACPI_BATTERY is not set CONFIG_ACPI_BUTTON=y CONFIG_ACPI_FAN=y CONFIG_ACPI_PROCESSOR=y # CONFIG_ACPI_THERMAL is not set # CONFIG_ACPI_ASUS is not set # CONFIG_ACPI_TOSHIBA is not set # CONFIG_ACPI_DEBUG is not set # CONFIG_ACPI_RELAXED_AML is not set Thanks. I hope this helps. -- Guus Houtzager Email: [EMAIL PROTECTED] PGP fingerprint = 5E E6 96 35 F0 64 34 14 CC 03 2B 36 71 FB 4B 5D A)bort, R)etry, I)nfluence with large hammer. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: SMP on Debian server with Hyperthreading
Jason Lim, Sat, Sep 06, 2003 at 01:06:43AM +0800: Just wondering... I've got a 2.4Ghz Hyperthreading (100% it is the hyperthreading model), and the BIOS sees it. I then compiled the kernel... the usual, except added the SMP support setting Symmetric multi-processing support. Nothing else was changed. Hope you can advise... as hyperthreading is there but not being used, which is a waste and could add performance. We have several dual Xeon machines with HT. If we use a standard smp kernel (there are several in debian, although we compile our own), the kernel will see 4 processors. If we turn off HT, the kernel will see two processors. We usually turn it off, as it doesn't provide significant performance gains to our application. Hopefully 2.6 will be able to take better advantage of it. I have not tried, however, to enable HT on a single proc P4 to see if the kernel sees 2 cpus. I don't remember HT being an option on those machines, although they have the same motherboard as the dual procs. In either case, you'll need to test your application with HT on and off and see how it affects performance. g pgp0.pgp Description: PGP signature
SMP on Debian server with Hyperthreading
Hi all, Just wondering... I've got a 2.4Ghz Hyperthreading (100% it is the hyperthreading model), and the BIOS sees it. I then compiled the kernel... the usual, except added the SMP support setting Symmetric multi-processing support. Nothing else was changed. Compiled it, liloed it... it's running it: # uname -a Linux megalith 2.4.22 #8 SMP Thu Aug 28 14:44:13 HKT 2003 i686 unknown However, # mpstat -P Not an SMP machine... And in top i don't see the multiple CPU usage this is all strange. For Linux, aren't Hyperthreading CPUs suppose to act like completely separate independent CPUs (this was suppose to change in 2.6... but for 2.4, they can't tell the difference, right?). Hope you can advise... as hyperthreading is there but not being used, which is a waste and could add performance. Thanks in advance! Jas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: SMP on Debian server with Hyperthreading
Just thought I'd add a bit more info... from dmesg: Linux version 2.4.22 ([EMAIL PROTECTED]) (gcc version 2.95.4 20011002 (Debian prerelease )) #8 SMP Thu Aug 28 14:44:13 HKT 2003 BIOS-provided physical RAM map: BIOS-e820: - 0009fc00 (usable) BIOS-e820: 0009fc00 - 000a (reserved) BIOS-e820: 000e6000 - 0010 (reserved) BIOS-e820: 0010 - 3f73 (usable) BIOS-e820: 3f73 - 3f74 (ACPI data) BIOS-e820: 3f74 - 3f7f (ACPI NVS) BIOS-e820: 3f7f - 3f80 (reserved) BIOS-e820: fecf - fecf1000 (reserved) BIOS-e820: fed2 - feda (reserved) 119MB HIGHMEM available. 896MB LOWMEM available. found SMP MP-table at 000ff780 hm, page 000ff000 reserved twice. hm, page 0010 reserved twice. hm, page 000fc000 reserved twice. hm, page 000fd000 reserved twice. On node 0 totalpages: 259888 zone(0): 4096 pages. zone(1): 225280 pages. zone(2): 30512 pages. Intel MultiProcessor Specification v1.4 Virtual Wire compatibility mode. OEM ID: Product ID: Springdale-G APIC at: 0xFEE0 Processor #0 Pentium 4(tm) XEON(tm) APIC version 20 I/O APIC #2 Version 32 at 0xFEC0. Enabling APIC mode: Flat. Using 1 I/O APICs Processors: 1 Kernel command line: auto BOOT_IMAGE=Linux ro root=801 Initializing CPU#0 Seems it KNOWS it is SMP... but then it detects only 1 processor? Is this how hyperthreading works? - Original Message - From: Jason Lim [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Saturday, 06 September, 2003 1:06 AM Subject: SMP on Debian server with Hyperthreading Hi all, Just wondering... I've got a 2.4Ghz Hyperthreading (100% it is the hyperthreading model), and the BIOS sees it. I then compiled the kernel... the usual, except added the SMP support setting Symmetric multi-processing support. Nothing else was changed. Compiled it, liloed it... it's running it: # uname -a Linux megalith 2.4.22 #8 SMP Thu Aug 28 14:44:13 HKT 2003 i686 unknown However, # mpstat -P Not an SMP machine... And in top i don't see the multiple CPU usage this is all strange. For Linux, aren't Hyperthreading CPUs suppose to act like completely separate independent CPUs (this was suppose to change in 2.6... but for 2.4, they can't tell the difference, right?). Hope you can advise... as hyperthreading is there but not being used, which is a waste and could add performance. Thanks in advance! Jas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
