spilling ?
Thanks.
Avi
-Original Message-
From: Allen Wittenauer [mailto:a...@apache.org]
Sent: Monday, August 22, 2011 7:06 AM
To: common-user@hadoop.apache.org
Subject: Re: Hadoop cluster optimization
On Aug 21, 2011, at 7:17 PM, Michel Segel wrote:
Avi,
First why 32 bit OS?
You have
cluster optimization
On Aug 21, 2011, at 7:17 PM, Michel Segel wrote:
Avi,
First why 32 bit OS?
You have a 64 bit processor that has 4 cores hyper threaded looks like
8cpus.
With only 1.7gb of mem, there likely isn't much of a reason to use a
64-bit OS. The machines (as you point
. (You said you're running this on EC2... ) Note: I'm not an expert on
EC2. I guess I'm lucky that I have my own sandbox to play in... :-)
HTH
-Mike
Date: Mon, 22 Aug 2011 13:00:27 +0300
Subject: Re: Hadoop cluster optimization
From: avivakni...@gmail.com
To: common-user@hadoop.apache.org
Hi
On Aug 22, 2011, at 3:00 AM, אבי ווקנין wrote:
I assumed that the 1.7GB RAM will be the bottleneck in my environment that's
why I am trying to change it now.
I shut down the 4 datanodes with 1.7GB RAM (Amazon EC2 small instance) and
replaced them with
2 datanodes with 7.5GB RAM (Amazon
this helps
-Original Message-
From: Avi Vaknin [mailto:avivakni...@gmail.com]
Sent: 2011年8月21日 快下班了 7:57
To: common-user@hadoop.apache.org
Subject: Hadoop cluster optimization
Hi all !
How are you?
My name is Avi and I have been fascinated by Apache Hadoop for the last few
months.
I
Avi,
First why 32 bit OS?
You have a 64 bit processor that has 4 cores hyper threaded looks like 8cpus.
With only 1.7 GB you're going to be limited on the number of slots you can
configure.
I'd say run ganglia but that would take resources away from you. It sounds
like the default parameters
On Aug 21, 2011, at 7:17 PM, Michel Segel wrote:
Avi,
First why 32 bit OS?
You have a 64 bit processor that has 4 cores hyper threaded looks like 8cpus.
With only 1.7gb of mem, there likely isn't much of a reason to use a
64-bit OS. The machines (as you point out) are already tight
