Fair Share Scheduler not worked as expected

2012-03-20 Thread WangRamon
Hi All I noticed there is something strange in my Fair Share Scheduler monitor GUI, the SUMof the Faire Share Value is always about 30 even there is only one M/R Job is running, so I don't know whether the value is about the usage percentage, if it was the percentage, that explains why all

Can Hadoop 1.0.1 work with HBase 0.90.5?

2012-03-17 Thread WangRamon
Hi all I'm looking for a compatible version with HBase 0.90.5, as i know Hadoop 1.0.0 can work with HBase, it has HBase (append/hsynch/hflush, and security) in its release note, but there is no such note in the Hadoop 1.0.1 release, so can anybody give me a correct answer? Thanks. BTW, i see

What is the NEW api?

2012-03-11 Thread WangRamon
Hi all I've been with Hadoop-0.20-append for a few time and I plan to upgrade to 1.0.0 release, but i find there are many people taking about the NEW API, so I'm lost, can anyone please tell me what is the new API? Is the OLD one available in the 1.0.0 release? Thanks CheersRamon

RE: does hadoop always respect setNumReduceTasks?

2012-03-10 Thread WangRamon
Jane, i think you have mapred.tasktracker.reduce.tasks.maximum or mapred.reduce.tasks set to 1 in your local, and have them set to some other values in the emr, that's why you always get one reducer in your local and not on the emr. CheersRamon Date: Thu, 8 Mar 2012 21:30:26 -0500 Subject:

Why most of the free reduce slots are NOT used for my Hadoop Jobs? Thanks.

2012-03-10 Thread WangRamon
Sorry for the duplicate, i have sent this mail to map/reduce mail list but haven't got any useful response yet, so i think maybe i can get some suggestions here, thanks. Hi All I'm using Hadoop-0.20-append, the cluster contains 3 nodes, for each node I have 14 map and 14 reduce slots, here

RE: How to manage hadoop job submit?

2011-11-21 Thread WangRamon
(org.apache.hadoop.mapreduce.JobPriority) [1 - Stable API] - http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred/JobConf.html#setJobPriority(org.apache.hadoop.mapred.JobPriority) HTH. On 20-Nov-2011, at 3:14 PM, WangRamon wrote: Hi All I'm new to hadoop, I know I

How to manage hadoop job submit?

2011-11-20 Thread WangRamon
Hi All I'm new to hadoop, I know I can use haddop jar to submit my M/R job, but we need to submit a lot of jobs in my real environment, there is priority requirement for each jobs, so is there any way to manage how to submit jobs? Any Java API? Or we can only use the hadoop command line with

RE: How to manage hadoop job submit?

2011-11-20 Thread WangRamon
] - Original Message - From: WangRamon ramon_w...@hotmail.com To: common-user@hadoop.apache.org Cc: Sent: Sunday, November 20, 2011 4:44 AM Subject: How to manage hadoop job submit? Hi All I'm new to hadoop, I know I can use haddop jar to submit my M/R job, but we need to submit a lot