Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Is there anybody knowing about this, please? On Mon, Jun 14, 2010 at 10:21 PM, Kevin Tse kevintse.on...@gmail.comwrote: Hi Ted, I mean the new API: org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat) Job.setInputFormatClass() only accepts org.apache.hadoop.mapreduce.InputFormat(of which there are several subclasses, while KeyValueTextInputFormat is not one of them) as its parameter. On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu yuzhih...@gmail.com wrote: Have you checked src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ? On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse kevintse.on...@gmail.com wrote: Hi, I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the process I found that there was no KeyValueTextInputFormat class which exists in hadoop-0.19.2. It's so strange that this version of hadoop does not come with this commonly used InputFormat. I have taken a look at the SecondarySort.java example code, it uses TextInputFormat and StringTokenizer to split each line, it is ok but kinda awkward to me. Do I have to implement a new InputFormat myself or there's a KeyValueTextInputFormat that exists somewhere I didn't notice? Thank you. Kevin Tse
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
The new api KeyValueTextInputFormat is not available in branch 0.20. It is added in branch 0.21 through https://issues.apache.org/jira/browse/MAPREDUCE-655. Thanks Amareshwari On 6/21/10 6:52 AM, Kevin Tse kevintse.on...@gmail.com wrote: Is there anybody knowing about this, please? On Mon, Jun 14, 2010 at 10:21 PM, Kevin Tse kevintse.on...@gmail.comwrote: Hi Ted, I mean the new API: org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat) Job.setInputFormatClass() only accepts org.apache.hadoop.mapreduce.InputFormat(of which there are several subclasses, while KeyValueTextInputFormat is not one of them) as its parameter. On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu yuzhih...@gmail.com wrote: Have you checked src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ? On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse kevintse.on...@gmail.com wrote: Hi, I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the process I found that there was no KeyValueTextInputFormat class which exists in hadoop-0.19.2. It's so strange that this version of hadoop does not come with this commonly used InputFormat. I have taken a look at the SecondarySort.java example code, it uses TextInputFormat and StringTokenizer to split each line, it is ok but kinda awkward to me. Do I have to implement a new InputFormat myself or there's a KeyValueTextInputFormat that exists somewhere I didn't notice? Thank you. Kevin Tse
No KeyValueTextInputFormat in hadoop-0.20.2?
Hi, I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the process I found that there was no KeyValueTextInputFormat class which exists in hadoop-0.19.2. It's so strange that this version of hadoop does not come with this commonly used InputFormat. I have taken a look at the SecondarySort.java example code, it uses TextInputFormat and StringTokenizer to split each line, it is ok but kinda awkward to me. Do I have to implement a new InputFormat myself or there's a KeyValueTextInputFormat that exists somewhere I didn't notice? Thank you. Kevin Tse
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Have you checked src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ? On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse kevintse.on...@gmail.com wrote: Hi, I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the process I found that there was no KeyValueTextInputFormat class which exists in hadoop-0.19.2. It's so strange that this version of hadoop does not come with this commonly used InputFormat. I have taken a look at the SecondarySort.java example code, it uses TextInputFormat and StringTokenizer to split each line, it is ok but kinda awkward to me. Do I have to implement a new InputFormat myself or there's a KeyValueTextInputFormat that exists somewhere I didn't notice? Thank you. Kevin Tse
Re: No KeyValueTextInputFormat in hadoop-0.20.2?
Hi Ted, I mean the new API: org.apache.hadoop.mapreduce.Job.setInputFormatClass(org.apache.hadoop.mapreduce.InputFormat) Job.setInputFormatClass() only accepts org.apache.hadoop.mapreduce.InputFormat(of which there are several subclasses, while KeyValueTextInputFormat is not one of them) as its parameter. On Mon, Jun 14, 2010 at 10:03 PM, Ted Yu yuzhih...@gmail.com wrote: Have you checked src/mapred/org/apache/hadoop/mapred/KeyValueTextInputFormat.java ? On Mon, Jun 14, 2010 at 6:51 AM, Kevin Tse kevintse.on...@gmail.com wrote: Hi, I am upgrading my code from hadoop-0.19.2 to hadoop-0.20.2, during the process I found that there was no KeyValueTextInputFormat class which exists in hadoop-0.19.2. It's so strange that this version of hadoop does not come with this commonly used InputFormat. I have taken a look at the SecondarySort.java example code, it uses TextInputFormat and StringTokenizer to split each line, it is ok but kinda awkward to me. Do I have to implement a new InputFormat myself or there's a KeyValueTextInputFormat that exists somewhere I didn't notice? Thank you. Kevin Tse
