Re: Working of combiner in hadoop
On Fri, Jul 4, 2014 at 10:59 AM, Chhaya Vishwakarma < chhaya.vishwaka...@lntinfotech.com> wrote: > Hi, > > > > If have two map tasks running on one node , i have written combiner class > also. > > Will combiner be called once for each map task or just once for both the > map tasks > > > > Can i write a logic inside map which will work as combiner ? if yes will > there be any side effect? > > > > > Hi Chaya, Refer the following URLS http://java.dzone.com/articles/designing-mapreduce-algorithms http://isaacslavitt.com/2014/01/01/in-mapper-combiner-pattern-for-mapreduce/ http://alpinenow.com/blog/in-mapper-combiner/ Best regards -- ** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in
Re: Working of combiner in hadoop
The key/value pairs are processes by the mapper independently of each other. The combiner logic deals with all the outputs from multiple key/value pairs do that logic can not be in the map method. On Jul 4, 2014 1:29 AM, "Chhaya Vishwakarma" < chhaya.vishwaka...@lntinfotech.com> wrote: > Hi, > > > > If have two map tasks running on one node , i have written combiner class > also. > > Will combiner be called once for each map task or just once for both the > map tasks > > > > Can i write a logic inside map which will work as combiner ? if yes will > there be any side effect? > > > > Regards, > > Chhaya Vishwakarma > > > > -- > The contents of this e-mail and any attachment(s) may contain confidential > or privileged information for the intended recipient(s). Unintended > recipients are prohibited from taking action on the basis of information in > this e-mail and using or disseminating the information, and must notify the > sender and delete it from their system. L&T Infotech will not accept > responsibility or liability for the accuracy or completeness of, or the > presence of any virus or disabling code in this e-mail" >
Working of combiner in hadoop
Hi, If have two map tasks running on one node , i have written combiner class also. Will combiner be called once for each map task or just once for both the map tasks Can i write a logic inside map which will work as combiner ? if yes will there be any side effect? Regards, Chhaya Vishwakarma The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
