How ones application(s) work is really the main factor about what type of compute system they should run on. Unfortunately we all don't have money for big SMP machines. With your modest budget for 10 to 15 compute nodes you would be best served by sticking with GigEthernet for your interconnect between machines. Depending upon how chatty your parallel applications are among one another the throughput that GigE provides is generally good enough for such a small machine count. If you do choose to look at high speed interconnects, such as Myrinet or InfiniBand, look to easily add $3000 per node to the final cost of your solution.
I would recommend not buying the latest greatest processors when spec'ing out your cluster in order to get the most bang for your buck. When I was a cluster buying customer I would limit myself to $1000/cpu. I often try to make the same recommendation to people buying new clusters, especially if it is their first cluster. Furthermore, if I was going to build a cluster today I would be hard pressed not to strongly look at the AMD Opteron platform. In 32bit mode they are as fast, if not faster, than most XEON procs and give you the ability to migrate your code over to 64bit when you see fit. But don't take my word for it, most vendors will be more than will to let you benchmark your codes on various types of system architectures if you ask. File serving can easily be done by a linux box via NFS on this size of cluster. NFS performance will degrade I/O bound applications so they should use a local scratch disk when ever possible. I'd also recommend looking at IDE to SCSI disk arrays as they provide a ton (ok, on the order of 4 Terabytes) of storage in 4U of space, yet yield some decent Read/Write speeds of 60 MB/S and won't break your budget. When buying a cluster you must not forget to take into account the machine room requirements it will have. Do not under estimate the amount of heat that even 10 dual processor nodes can produce if not properly cooled. Manageability of the system must be considered. If possible try to budget in a serial console terminal server and some intelligent PDUs as they will save you time and headache when things break. A nice server deployment system such as FAI, Oscar, ROCKS or warewulf-cluster will save the administrator and scientists time when servers fail. happy shopping.. -mike -- [EMAIL PROTECTED] cell: 516.410.4478 https://secure.hanulec.com EFnet irc && aol im: hanulec On Wed, 3 Dec 2003, Ross Boylan wrote: > Although this list seems to have been quiet recently, perhaps there are > some folks out there with wisdom to share. I didn't turn up much in the > archives. > > The group I am in is about to purchase a cluster. If anyone on this > list has any advice on what type of hardware (or software) would be > best, I'd appreciate it. > > We will have two broad types of uses: simulation studies for > epidemiology (with people or cases as the units) and genetic and protein > studies, with which I am less familiar. The simulation studies are > likely to make heavy use of R. I suspect that the two uses have much > different characteristics, e.g., in terms of the size of the datasets to > manipulate and the best tradeoffs outlined below. > > Other uses are possible. > > Among other issues we are wondering about: > *Tradeoffs between CPU speed, memory, internode communication speed, > disk size, and disk speed. > > As a first cut, I expect the simulations suggest emphasizing processor > power and ensuring adequate memory. On the other hand, the fact that > it's easy to upgrade CPUs suggests putting more money into the network > supporting the CPUs. And I suspect the genomics emphasizes more the > ability to move large amounts of data around quickly (across network and > to disk). > > *Appropriate disk architecture (e.g., local disks vs shared netword > disks or SANS). > > 32 vs 64 bit; Intel vs AMD. > > We assume it will be some kind of Linux OS (we like Debian, but vendors > tend to supply RH and Debian lacks support for 64 bit AMD in any > official way, unlike Suse or RH). If there's a good reason, we could > use something else. > > Our budget is relatively modest, enough perhaps for 10-15 dual-processor > nodes. We hope to expand later. > > As a side issue, more a personal curiosity, why do clusters all seem to > be built on dual-processor nodes? Why not more CPU's per node? > > Thanks for any help you can offer. > -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
