Hello, I am doing hadoop testing on the harmony select(r946981) and when I try to run some application, I encountered some problems.Maybe some defect exists inside the harmony select caused these problems and I will be very thankful if anybody could give me some advise to solve it. I have configured 4 machine hadoop cluster(1 namenode, 1 jobtracker, 2 slaves), on which hadoop-0.20.2 could be started up successfully. All of the 4 machines are linux(ubuntu) platform, JAVA_HOME is set to be the path of harmony select, other configuration values are just set to be default. After the hadoop cluster is started up, I ran the terasort, a sorting application, with 500M input data. During the testing, the job failed. console messageļ¼ 10/07/02 13:23:44 INFO terasort.TeraSort: starting 10/07/02 13:23:45 INFO mapred.FileInputFormat: Total input paths to process : 2 10/07/02 13:23:47 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 10/07/02 13:23:47 INFO compress.CodecPool: Got brand-new compressor Making 1 from 100000 records Step size is 100000.0 10/07/02 13:23:50 INFO mapred.JobClient: Running job: job_201007021322_0001 10/07/02 13:23:51 INFO mapred.JobClient: map 0% reduce 0% 10/07/02 13:24:43 INFO mapred.JobClient: map 7% reduce 0% 10/07/02 13:24:46 INFO mapred.JobClient: map 12% reduce 0% 10/07/02 13:25:06 INFO mapred.JobClient: map 13% reduce 0% 10/07/02 13:25:09 INFO mapred.JobClient: map 15% reduce 0% 10/07/02 13:25:12 INFO mapred.JobClient: map 16% reduce 0% 10/07/02 13:25:15 INFO mapred.JobClient: map 18% reduce 0% 10/07/02 13:25:18 INFO mapred.JobClient: map 19% reduce 0% 10/07/02 13:25:21 INFO mapred.JobClient: map 20% reduce 0% 10/07/02 13:25:24 INFO mapred.JobClient: map 22% reduce 0% 10/07/02 13:25:27 INFO mapred.JobClient: map 23% reduce 0% 10/07/02 13:25:30 INFO mapred.JobClient: map 24% reduce 0% 10/07/02 13:34:55 INFO mapred.JobClient: map 12% reduce 0% 10/07/02 13:35:03 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000002_0, Status : FAILED Task attempt_201007021322_0001_m_000002_0 failed to report status for 601 seconds. Killing! 10/07/02 13:35:03 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000003_0, Status : FAILED Task attempt_201007021322_0001_m_000003_0 failed to report status for 602 seconds. Killing! 10/07/02 13:35:16 INFO mapred.JobClient: map 17% reduce 0% 10/07/02 13:35:19 INFO mapred.JobClient: map 23% reduce 0% 10/07/02 13:35:22 INFO mapred.JobClient: map 24% reduce 0% 10/07/02 13:35:46 INFO mapred.JobClient: map 12% reduce 0% 10/07/02 13:36:06 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000000_0, Status : FAILED Task attempt_201007021322_0001_m_000000_0 failed to report status for 602 seconds. Killing! 10/07/02 13:36:07 INFO mapred.JobClient: Task Id : attempt_201007021322_0001_m_000001_0, Status : FAILED Task attempt_201007021322_0001_m_000001_0 failed to report status for 600 seconds. Killing! 10/07/02 13:36:30 INFO mapred.JobClient: map 13% reduce 0% 10/07/02 13:36:33 INFO mapred.JobClient: map 16% reduce 0% 10/07/02 13:36:36 INFO mapred.JobClient: map 18% reduce 0% ....
log : 2010-07-02 13:35:09,538 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=MAP, sessionId= 2010-07-02 13:35:10,433 INFO org.apache.hadoop.mapred.MapTask: numReduceTasks: 1 2010-07-02 13:35:10,459 INFO org.apache.hadoop.mapred.MapTask: io.sort.mb = 100 2010-07-02 13:35:11,073 INFO org.apache.hadoop.mapred.MapTask: data buffer = 79691776/99614720 2010-07-02 13:35:11,074 INFO org.apache.hadoop.mapred.MapTask: record buffer = 262144/327680 2010-07-02 13:35:11,308 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2010-07-02 13:35:11,311 INFO org.apache.hadoop.io.compress.CodecPool: Got brand-new decompressor 2010-07-02 13:35:16,153 INFO org.apache.hadoop.mapred.MapTask: Spilling map output: record full = true 2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 26214400; bufvoid = 99614720 2010-07-02 13:35:16,154 INFO org.apache.hadoop.mapred.MapTask: kvstart = 0; kvend = 262144; length = 327680 Look forward to any reply. Best Regards, Cindy
