On Wed, 9 Apr 2008, Chris K Wensel wrote:
make sure all nodes are running in the same 'availability zone', http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1347

check!

and that you are using the new xen kernels.
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1353&categoryID=101
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1354&categoryID=101

check!

also, make sure each node is addressing its peers via the ec2 private addresses, not the public ones.

check!

there is a patch in jira for the ec2/contrib scripts that address these issues.
https://issues.apache.org/jira/browse/HADOOP-2410

if you use those scripts, you will be able to see a ganglia display showing utilization on the machines. 8/7 map/reducers sounds like alot.

Reduced - I dropped it to 3/2 for testing.

I am using these scripts now, and am still seeing very poor performance on EC2 compared to my development environment. ;(

I'll be capturing some more extensive stats over the weekend, and see if I can glean anything useful...

------------------------------------------------------------------------
| nate carlson | [EMAIL PROTECTED] | http://www.natecarlson.com |
|       depriving some poor village of its idiot since 1981            |
------------------------------------------------------------------------

Reply via email to