Hi Harsh,
I resolved the earlier issue I had with mapper but the configure is not
getting called.
The signature for Mapper is
public void map(LongWritable key, Text value, OutputCollector output,
Reporter reporter) throws IOException {
I used @override does not seem to work.
Configure is not getting called as I populate some collections there
that are used later in the mapper and I am getting those collections as
blank in the mapper.
Regards,
Shreya
From: Harsh J [mailto:ha...@cloudera.com]
Sent: Thursday, September 15, 2011 5:14 PM
To: mapreduce-user@hadoop.apache.org
Subject: Re: Error calling config method in Mapper
Hello again Shreya,
That signature (for configure) looks right and I think it indeed gets
called. Unsure about your map call issue as I do not see that signature.
Perhaps what you need to resolve this issue by yourself is to make use
of the @Override method annotations in your code. That helps a lot when
dealing with frameworks, such as Hadoop.
On Thu, Sep 15, 2011 at 4:40 PM, <shreya....@cognizant.com> wrote:
Hi
I have written a mapper class that has the following structure
public class TestMapper extends MapReduceBase implements
Mapper<LongWritable, Text, Text, Text> {
public void configure(JobConf conf) {
Path [] cacheFiles = DistributedCache.getLocalCacheFiles(conf);
if (null != cacheFiles && cacheFiles.length > 0) {
for (Path cachePath : cacheFiles) {
if (cachePath.getName().equals(stopwordCacheName)) {
loadStopWords(cachePath);
break;
}
if (cachePath.getName().equals(positiveCacheName)) {
loadPositiveWords(cachePath);
break;
}
if (cachePath.getName().equals(negativeCacheName)) {
loadNegativeWords(cachePath);
break;
}
if (cachePath.getName().equals(searchCacheName)) {
loadSearchWords(cachePath);
break;
}
}
//Mapper logic goes here
}
Config method should get called before the mapper method by default
right?
But that's not happening..... and the loggers I have used inside mapper
are not getting printed on console and there are no loga in userlogs.
Userlogs is empty. Logger
statements(sLogger.debug("*****************uploaded positive words");)
in the driver class are getting printed on the console though.
Regards,
Shreya
This e-mail and any files transmitted with it are for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If you are not the intended recipient(s), please reply to
the sender and destroy all copies of the original message. Any
unauthorized review, use, disclosure, dissemination, forwarding,
printing or copying of this email, and/or any action taken in reliance
on the contents of this e-mail is strictly prohibited and may be
unlawful.
--
Harsh J
This e-mail and any files transmitted with it are for the sole use of the
intended recipient(s) and may contain confidential and privileged information.
If you are not the intended recipient(s), please reply to the sender and
destroy all copies of the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email,
and/or any action taken in reliance on the contents of this e-mail is strictly
prohibited and may be unlawful.
