Hi all, I am trying to build a Hadoop/MR application in c++ using hadoop-pipes. I have been able to successfully work with my own mappers and reducers, but now I need to generate output (from reducer) in a format different from the default TextOutputFormat. I have a few questions:
(1) Similar to Hadoop streaming, is there an option to set OutputFormat in HadoopPipes (in order to use say org.apache.hadoop.io.SequenceFile.Writer) ? I am using Hadoop version 0.20.2. (2) For a simple test on how to use an in-built non-default writer, I tried the following: hadoop pipes -D hadoop.pipes.java.recordreader=true -D hadoop.pipes.java.recordwriter=false -input input.seq -output output -inputformat org.apache.hadoop.mapred.SequenceFileInputFormat -writer org.apache.hadoop.io.SequenceFile.Writer -program my_test_program However this fails with a ClassNotFound exception. And if I remove the -writer flag and use the default writer, it works just fine. (3) Is there some example or discussion related to how to write your own RecordWriter and run it with Hadoop-pipes ? Thanks. Best, Vivek --