RE: Enable Hyperthreading? / Typical M/R ratios and rules of thumb

2011-11-16 Thread Jeffrey Buell
> > How much RAM do you have? > > > > A good rule of thumb is to use 1-1.5G for maps and 2G per reduce > > (vmem). Ensure your OS has at least 2G of memory. > > > > Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale > up > > if you have more memory. > > Would you say RAM was the m

Re: Enable Hyperthreading? / Typical M/R ratios and rules of thumb

2011-11-16 Thread Tom Hall
Thanks Arun, On Mon, Nov 14, 2011 at 4:34 AM, Arun Murthy wrote: > How much RAM do you have? > > A good rule of thumb is to use 1-1.5G for maps and 2G per reduce > (vmem). Ensure your OS has at least 2G of memory. > > Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale up > if you

Re: Enable Hyperthreading? / Typical M/R ratios and rules of thumb

2011-11-13 Thread Arun Murthy
How much RAM do you have? A good rule of thumb is to use 1-1.5G for maps and 2G per reduce (vmem). Ensure your OS has at least 2G of memory. Thus, with 24G and dual quad cores you should be at 8-10m/2r. Scale up if you have more memory. Also, ensure you turn on memory monitoring so that a rouge

Enable Hyperthreading? / Typical M/R ratios and rules of thumb

2011-11-13 Thread Tom Hall
Hi All, We have been using dual socket quad core machines for a while and have been running with 8 mappers 2 reducers. The rule of thumb we heard was slightly oversubscribe the number of cores but at Hadoop World several people said other things. Some new machines we are moving to have 2 socket 6