I believe that streaming still only uses the older mapred API and does not
support the API in the mapreduce java package.
--Bobby Evans
On 1/4/12 11:07 AM, "Kadu canGica Eduardo" <kaducang...@gmail.com> wrote:
Hi,
i'm using streaming with python and i made my own inputformat using the new API
but when i run my job i got the error message:
"org.fasta.InputFormat.FastaInputFormat not
org.apache.hadoop.mapred.InputFormat".
Well, as far as i know the new API was introduced in 0.20 (i'm using
0.20.203.0), but it seems that streaming don't work with it.
Is there any way to fix this without rewriting all my code of inputformat with
the old API?
Thanks in advance.
Carlos.
package org.fasta.InputFormat;
import java.io.IOException;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.compress.CompressionCodec;
import org.apache.hadoop.io.compress.CompressionCodecFactory;
import org.apache.hadoop.mapreduce.InputSplit;
import org.apache.hadoop.mapreduce.JobContext;
import org.apache.hadoop.mapreduce.RecordReader;
import org.apache.hadoop.mapreduce.TaskAttemptContext;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
public class FastaInputFormat extends FileInputFormat<LongWritable,Text> {
public static String addFistQualityValueProperty =
"fastainputformat.addFistQualityValue";
@Override
protected boolean isSplitable(JobContext context, Path file) {
CompressionCodec codec = new CompressionCodecFactory(context
.getConfiguration()).getCodec(file);
return codec == null;
}
@Override
public RecordReader<LongWritable, Text> createRecordReader(InputSplit split,
TaskAttemptContext context) throws IOException,
InterruptedException {
return new FastaRecordReader();
}
}
