Currently the DistributedCache is populated pre-Job run, hence both Map and Reduce phases carry the same items. With MR2, the approach Robert describes above should work better instead.
On Sat, Apr 21, 2012 at 5:21 AM, JAX <jayunit...@gmail.com> wrote: > No reducers can't access mapper counters. > ---> maybe theres a way to intermediately put counters in the distributed > cache??? > > Jay Vyas > MMSB > UCHC > > On Apr 20, 2012, at 1:24 PM, Robert Evans <ev...@yahoo-inc.com> wrote: > >> There was a discussion about this several months ago >> >> http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201112.mbox/%3CCADYHM8xiw8_bF=zqe-bagdfz6r3tob0aof9viozgtzeqgkp...@mail.gmail.com%3E >> >> The conclusion is that if you want to read them from the reducer you are >> going to have to do something special until someone finds time to implement >> it as part of. >> >> https://issues.apache.org/jira/browse/MAPREDUCE-3520 >> >> --Bobby Evans >> >> >> On 4/20/12 11:36 AM, "Amith D K" <amit...@huawei.com> wrote: >> >> Yes U can use user defined counter as Jagat suggeted. >> >> Counter can be enum as Jagat described or any string which are called >> dynamic counters. >> >> It is easier to use Enum counter than dynamic counters, finally it depends >> on your use case :) >> >> Amith >> ________________________________________ >> From: Jagat [jagatsi...@gmail.com] >> Sent: Saturday, April 21, 2012 12:25 AM >> To: common-user@hadoop.apache.org >> Subject: Re: Accessing global Counters >> >> Hi >> >> You can create your own counters like >> >> enum CountFruits { >> Apple, >> Mango, >> Banana >> } >> >> >> And in your mapper class when you see condition to increment , you can use >> Reporter incrCounter method to do the same. >> >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/Reporter.html#incrCounter(java.lang.Enum,%20long) >> >> e.g >> // I saw Apple increment it by one >> reporter.incrCounter(CountFruits.Apple,1); >> >> Now you can access them using job.getCounters >> >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#getCounters() >> >> Hope this helps >> >> Regards, >> >> Jagat Singh >> >> >> On Fri, Apr 20, 2012 at 9:43 PM, Gayatri Rao <rgayat...@gmail.com> wrote: >> >>> Hi All, >>> >>> Is there a way for me to set global counters in Mapper and access them from >>> reducer? >>> Could you suggest how I can acheve this? >>> >>> Thanks >>> Gayatri >>> >> -- Harsh J
