Yes, the map outputs will be cleaned on job completion. If u want to see the map outputs give number of reducers as zero and verify the files part-m-0000, part-m-0001....
On 12/16/14, bit1...@163.com <bit1...@163.com> wrote: > Do they only exist during the map/reduce process and will be removed after > the MR finished？ > > When the reduce finished，I only see part-m-0000, part-m-0001 ...., which > are reduce results. > > > > bit1...@163.com > > From: Susheel Kumar Gadalay > Date: 2014-12-16 13:05 > To: user > Subject: Re: Where the output of mappers are saved ? > Map outputs will be in hdfs under your user name and output directory. > > They will have name like part-m-0000, part-m-0001 .... > > > On 12/16/14, Abdul Navaz <navaz....@gmail.com> wrote: >> Hello, >> >> >> Second Try ! >> >> >> I have created a directory to store this mapper output as below. >> <property> >> <name>mapred.local.dir</name> >> <value>/app/hadoop/tmp/myoutput</value> >> </property> >> and i looked at >> hduser@dn4:/app/hadoop/tmp/myoutput$ ls -lrt >> total 16 >> drwxr-xr-x 2 hduser hadoop 4096 Dec 12 10:50 tt_log_tmp >> drwx------ 3 hduser hadoop 4096 Dec 12 10:53 ttprivate >> drwxr-xr-x 3 hduser hadoop 4096 Dec 12 10:53 taskTracker >> drwxr-xr-x 4 hduser hadoop 4096 Dec 12 13:25 userlogs >> and i couldnot find anything here when i run the map reduce job . Where >> by >> default mapper output is saved and how can I get the size of mapper >> output >> in bytes >> >> >> Thanks. >> >> >> From: Abdul Navaz <navaz....@gmail.com> >> Date: Friday, December 12, 2014 at 12:36 AM >> To: "user@hadoop.apache.org" <user@hadoop.apache.org> >> Subject: Where the output of mappers are saved ? >> >> Hello, >> >> >> I am interested in efficiently manage the Hadoop shuffling traffic and >> utilize the network bandwidth effectively. To do this I want to know how >> much shuffling traffic generated by each Datanodes ? Shuffling traffic is >> nothing but the output of mappers. So where this mapper output is saved ? >> How can i get the size of mapper output from each datanodes in a real >> time >> ? >> Appreciate your help. >> >> Thanks & Regards, >> >> Abdul Navaz >> >> >> >> >
