Re: SMP Linux
On Thu, Jan 30, 2003 at 01:18:41PM +, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. IIRC, Linus is quite specific in his intention that linux serve the lower-end market where making linux efficient on 2 processors is the goal. More procs is theoretically interesting but practically does not have a significant userbase. Linux is not aiming at competition with supercomputers. It's been a while since I read kernel traffic but I think the scheduler is undergoing some work to help with more CPUs anyway though. I'd concur with Steve on NUMA development. Paul -- Paul Makepeace ... http://paulm.com/ What is the thing I'm thinking about now? There is no spoon. -- http://paulm.com/toys/surrealism/
Re: SMP Linux
On Thu, Jan 30, 2003 at 01:18:41PM +, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. I used to run it happily on 2 CPU's, but I wondered if anyone was doing it seriously on more than 4. I did some linux-on-zSeries[0] work on a many-way power machine. And some 4-way RS/6000 work too. I think Linux on 4-way Xeon is meant to be pretty good, but I don't know about benchmarks vs eg FreeBSD or Slowaris. /joel [0] - the server formerly known as S/390
Re: SMP Linux
On Thu, Jan 30, 2003 at 01:18:41PM +, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. I used to run it happily on 2 CPU's, but I wondered if anyone was doing it seriously on more than 4. I could be wrong on this, and I suspect that doop (Jon Chin) or quidity (Alex Gough) are better people to ask, but basically over about 4 processors SMP is a bad idea, and you start to want an architecture that is MPP-based, where you have effectively independent computers connected by a very fast network-like bus. I'm not sure what the big machines (E10k/E15k/S390/etc) use, but I would have thought that it's unlikely to be SMP, just due to the need to occasionally lock the bus, which scales appallingly badly. This is of course, the type of machine where Solaris really comes into its own, however. -- Lusercop.net - LARTing Lusers everywhere since 2002
Re: SMP Linux
From: Lusercop `the.lusercop'@lusercop.net On Thu, Jan 30, 2003 at 01:18:41PM +, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. I used to run it happily on 2 CPU's, but I wondered if anyone was doing it seriously on more than 4. Most of the stuff on the web suggests that linux doesn't (or didn't) scale particularly well past 4 processors on Intel and I doubt there are many people in the world running this since Solaris on UltraSPARC is more common in this niche. Although I think a lot of current linux development work is on SMP and multi processor scalability so if you were brave enough to run a bleeding edge kernel with dodgy patches you might get better results. And as ever in the linux world what I saying may be totally out of date as of last week. 2.6 is likely to have NUMA. I could be wrong on this, and I suspect that doop (Jon Chin) or quidity (Alex Gough) are better people to ask, but basically over about 4 processors SMP is a bad idea, and you start to want an architecture that is MPP-based, where you have effectively independent computers connected by a very fast network-like bus. I'm not sure what the big machines (E10k/E15k/S390/etc) use, but I would have thought that it's unlikely to be SMP, just due to the need to occasionally lock the bus, which scales appallingly badly. I suspect MPP (message passing) versus SMP (shared memory) is a CS religious issue and don't claim to understand all the issues but isn't MPP generally very slow? I suppose you have to draw tradeoff graphs. AFAIK the big Suns *do* use SMP and I can't think of any mainstream UNIX server kernel which uses MPP (maybe Alphas running OSF?). Although Apple's Darwin is based on Mach (and other things) there seems to be a bit of doubt about how well supported the Mach stuff is and they seem to use SMP for their multiprocessor support. What's better in theory often doesn't happen in practice since if the CS academics of a decade back had predicted future technology correctly then we would be all running microkernels on RISC chips today. And we aren't. -- 1024/D9C69DF9 Steve Mynott [EMAIL PROTECTED]
Re: SMP Linux
On Thu, Jan 30, 2003 at 05:53:49PM -, Steve Mynott wrote: Although I think a lot of current linux development work is on SMP and multi processor scalability so if you were brave enough to run a bleeding edge kernel with dodgy patches you might get better results. The sparc-kernel list has several people running 4 and 8-way boxes (someone was talking about a V880 last week). Don't know how it runs compared to Sol9 tho' -- Chris Benson
Re: SMP Linux
On Thu, 2003-01-30 at 13:18, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. I used to run it happily on 2 CPU's, but I wondered if anyone was doing it seriously on more than 4. Have a look at mosix (see: http://www.mosix.org/). Personally, although SMP is one of the major thrusts of 2.5 (thru finer grained locks, better cache coherency, better memory management etc), I think MPP is a better way to go. Seeing as these mini-itx boards are now so cheap and low powered (as in VAs, see: http://www.mini-itx.com/), I was toying with the idea of making myself a small cluster of these and running them off a humungous (60A) 12V PSU I have lying around not doing (many) Radio Ham duties these days. The boards seem only to consume about 5W each. I can see a nice (cool) server farm being made out of a bunch of these as well. Who needs 'blade servers' @ $800+ a pop? Dirk -- Please Note: Some Quantum Physics Theories Suggest That When the Consumer Is Not Directly Observing This Product, It May Cease to Exist or Will Exist Only in a Vague and Undetermined State.
Re: SMP Linux
On Thu, Jan 30, 2003 at 01:18:41PM +, Jonathan Peterson wrote: Anyone using Linux on anything with lots of CPU's? Attempt to do searches for 'linux smp' on google tends to get me documents last updated in 1997. I used to run it happily on 2 CPU's, but I wondered if anyone was doing it seriously on more than 4. Seen it run fine on 2 and 4 way Intel setups. Long periods of time doing moderately hard work. Haven't tried more than that.. Never really had a task that was suited to it, really. Just started spreading across more servers instead, which gets you other bonuses too. For 4 way, Intel architecture probably isn't the way to go. Moving on to something else -- I remember AMD claiming that the K7 stuff used EV6 interconnects or something, and so should work a lot better than the Pentium for SMP. Has anyone actually gone out and compared performance of 2,4,etc way setups? (Is there even 2-CPU AMD boards out there?) tjc -- Turning and turning in the widening gyre The falcon cannot hear the falconer; Things fall apart, the centre cannot hold; Mere anarchy is loosed upon the world.
