One or two 1gig nics on a 10g backbone sound reasonable with "only" 4 1T drives. 12*2T disks per node are getting more common and do not all have 10gig network cards, even on 600+ node clusters.
Cheers, Joep Sent from my iPhone On Dec 23, 2011, at 11:15 AM, Mads Toftum <m...@toftum.dk> wrote: > On Fri, Dec 23, 2011 at 12:23:59PM -0500, Koert Kuipers wrote: >> For a hadoop cluster that starts medium size (50 nodes) but could grow to >> hundred of nodes, what is the recommended network in the rack? 1gig or 10gig >> We have machines with 8 cores, 4 X 1tb drive (could grow to 8 X 1b drive), >> 48 Gb ram per node. >> We expect "balanced" usage of the cluster (both storage and computations). > > If it were me, I'd start by looking at the amount of data you need to > handle. Guestimate on both peak and average size. For simplification, start > by > guessing whether you'll be reading or writing more. If writes will be > the driver, then you need to adjust for the number of copies. Or if you > don't know how much data, then you could do a read and write test on one > of the machines and figure out how much it can produce and go from that. > I haven't looked at the pricing of 10gig gear lately, but I wouldn't be > surprised if it made more sense for you to run simple nodes with a > couple of 1gig cards and your infrastructure / backbone on 10gig. > There's really too much ymmv based on your specific case, so that you'll > only really know if you test on a handfull of machines. > > vh > > Mads Toftum > -- > http://soulfood.dk
