Re: Which hardware to choose

2012-10-02 Thread Alexander Pivovarov
All configs are per node. No HBase, only Hive and Pig installed On Tue, Oct 2, 2012 at 9:40 PM, Michael Segel wrote: > I think he's saying that its 24 maps 8 reducers per node and at 48GB that > could be too many mappers. > Especially if they want to run HBase. > > On Oct 2, 2012, at 8:14 PM, had

Re: Which hardware to choose

2012-10-02 Thread Michael Segel
Ah that's the $64,000.00 (USD) question I tend to be conservative so this should be a good starting point. You start with 2 things... the amount of memory available and the number of physical cores. Subtract a core for each main process. e.g. DN, TT, and RS if you're running HBase. Tak

Re: Which hardware to choose

2012-10-02 Thread Marcos Ortiz
Which is a reasonable number in this hardware? On 10/02/2012 09:40 PM, Michael Segel wrote: I think he's saying that its 24 maps 8 reducers per node and at 48GB that could be too many mappers. Especially if they want to run HBase. On Oct 2, 2012, at 8:14 PM, hadoopman wrote: Only 24 map and

Re: Which hardware to choose

2012-10-02 Thread Michael Segel
I think he's saying that its 24 maps 8 reducers per node and at 48GB that could be too many mappers. Especially if they want to run HBase. On Oct 2, 2012, at 8:14 PM, hadoopman wrote: > Only 24 map and 8 reduce tasks for 38 data nodes? are you sure that's right? > Sounds VERY low for a clu

Re: Which hardware to choose

2012-10-02 Thread hadoopman
Had to ask :D On 10/02/2012 07:19 PM, Russell Jurney wrote: I believe he means per node. Russell Jurney http://datasyndrome.com On Oct 2, 2012, at 6:15 PM, hadoopman wrote: Only 24 map and 8 reduce tasks for 38 data nodes? are you sure that's right? Sounds VERY low for a cluster that si

Re: Which hardware to choose

2012-10-02 Thread Russell Jurney
I believe he means per node. Russell Jurney http://datasyndrome.com On Oct 2, 2012, at 6:15 PM, hadoopman wrote: > Only 24 map and 8 reduce tasks for 38 data nodes? are you sure that's right? > Sounds VERY low for a cluster that size. > > We have only 10 c2100's and are running I believe 140

Re: Which hardware to choose

2012-10-02 Thread hadoopman
Only 24 map and 8 reduce tasks for 38 data nodes? are you sure that's right? Sounds VERY low for a cluster that size. We have only 10 c2100's and are running I believe 140 map and 70 reduce slots so far with pretty decent performance. On 10/02/2012 12:55 PM, Alexander Pivovarov wrote: 38

Re: Which hardware to choose

2012-10-02 Thread Alexander Pivovarov
Not sure the following options are available Integrated ICH10R on motherboard LSI® 6Gb SAS2008 daughtercard Dell PERC H200 Dell PERC H700 LSI MegaRAID® SAS 9260-8i http://www.dell.com/us/enterprise/p/poweredge-c2100/pd On Tue, Oct 2, 2012 at 10:59 AM, Oleg Ruchovets wrote: > Great , > > Thank y