Hi, 

With several days investigation, the wordcount-nopipe example in the 
hadoop-0.19.1 package can be run finally. However, there are some changes I did 
but not sure if this is proper/correct way. Could anyone please help to verify?

1. start up the job with the -inputformat argument with value 
"org.apache.hadoop.mapred.pipes.WordCountInputFormat"
2. since the RecordReader/Writer in C++ is used, so no Java based 
RecordReader/Writer should be used. However, I faced the error like 
"RecordReader defined while not needed" from the pipes. After checking the 
org.apache.hadoop.mapred.pipes.Submitter.java, I found this code snippet: 

if (results.hasOption("-inputformat")) {
        setIsJavaRecordReader(job, true);
        job.setInputFormat(getClass(results, "-inputformat", job,
                                     InputFormat.class));
}

So it seems that with -inputformat specified, the JavaRecordReader will be 
enabled. This caused the error before. Then I comment the line 
"setIsJavaRecordReader(job, true);". Then the examples can be run. Is this the 
proper way to make the wordcount-nopipe example works? I see from the code that 
there is a commented line "//cli.addArgument("javareader", false, "is the 
RecordReader in Java");" . Should this line be uncommented to support the 
disable of JavaRecordReader in command line option?

3. It seems that the wordcount-nopipe only works for input/output with local 
URI  file:///home/...  Is this true?

Thanks,
Jianmin


      

Reply via email to