That is not really a bug. Only if you use @Override will you be really
asserting that you've overriden the right method (since new API uses
inheritance instead of interfaces). Without that kinda check, its easy
to make mistakes and add in methods that won't get considered by the
framework (and
: Fri, 3 Aug 2012 13:02:32
To: mapreduce-user@hadoop.apache.org
Reply-To: mapreduce-user@hadoop.apache.org
Cc: Mohammad Tariqdonta...@gmail.com
Subject: Re: Reading fields from a Text line
That is not really a bug. Only if you use @Override will you be really
asserting that you've overriden
2012 13:02:32
To: mapreduce-user@hadoop.apache.org
Reply-To: mapreduce-user@hadoop.apache.org
Cc: Mohammad Tariqdonta...@gmail.com
Subject: Re: Reading fields from a Text line
That is not really a bug. Only if you use @Override will you be really
asserting that you've overriden the right
Tariqdonta...@gmail.com
Subject: Re: Reading fields from a Text line
Bejoy,
In the new API, the default map() function, if not properly
overridden, is the identity map function. There is no IdentityMapper
class in the new API, the Mapper class itself is identity by default.
On Fri, Aug 3, 2012 at 1:07
Thanks for the response Harsh n Sri. Actually, I was trying to prepare
a template for my application using which I was trying to read one
line at a time, extract the first field from it and emit that
extracted value from the mapper. I have these few lines of code for
that :
public static class
Hi Tariq,
Is your file splittable? If it's not, Mapper will process entire file in
one go!
http://hadoop.apache.org/common/docs/r0.20.1/api/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.html#isSplitable%28org.apache.hadoop.mapreduce.JobContext,%20org.apache.hadoop.fs.Path%29
How many
on that as well.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-Original Message-
From: Mohammad Tariq donta...@gmail.com
Date: Thu, 2 Aug 2012 15:48:42
To: mapreduce-user@hadoop.apache.org
Reply-To: mapreduce-user@hadoop.apache.org
Subject: Re: Reading fields from a Text line
-user@hadoop.apache.org
Subject: Re: Reading fields from a Text line
Thanks for the response Harsh n Sri. Actually, I was trying to prepare
a template for my application using which I was trying to read one
line at a time, extract the first field from it and emit that
extracted value from
Hi Tariq
Again I strongly suspect the IdentityMapper in play here. The reasoning why
I suspect so is
When you have the whole data in output file it should be the Identity
Mapper. Due to the mismatch in input key type at class level and method
level the framework is falling back to
Hi Tariq
On further analysis I noticed a odd behavior in this context.
If we use the default InputFormat (TextInputFormat) but specify the Key
type in mapper as IntWritable instead of Long Writable. The framework is
supposed throw a class cast exception.Such an exception is thrown only if
the
Hello list,
I have a flat file in which data is stored as lines of 107
bytes each. I need to skip the first 8 lines(as they don't contain any
valuable info). Thereafter, I have to read each line and extract the
information from them, but not the line as a whole. Each line is
composed of
Mohammad,
But it seems I am not doing things in correct way. Need some guidance.
What do you mean by the above? What is your written code exactly
expected to do and what is it not doing? Perhaps since you ask for a
code question here, can you share it with us (pastebin or gists,
etc.)?
For
Wouldn't it be better if you could skip those unwanted lines
upfront(preprocess) and have a file which is ready to be processed by the
MR system? In any case, more details are needed.
On Thu, Aug 2, 2012 at 8:23 AM, Harsh J ha...@cloudera.com wrote:
Mohammad,
But it seems I am not doing
13 matches
Mail list logo
