when hadoop report job finished?
Hi all, I run terasort test on pseudo distributed, I found the one reduce job finished time is shorter than the four reduces job. I used tool to search why. Use nmon to monitor the disk read/write operation, I found a phenomena, when one reduce job report finished but the disk write data operation is still sustained for a long time. For four reduce jobs, when job report finished the disk read/write operation is done. In a nutshell, one reduce job finished time add disk write operation time equal to four reduces job finished time. So I think this is the reason why one reduce job time is shorter than the four reduces job, finally, I want to ask why that happened? What is the reduce job finished mark? The information and any attached documents contained in this message may be confidential and/or legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender immediately by return e-mail and destroy all copies of the original message.
when hadoop report job finished?
Hi all, I run terasort test on pseudo distributed, I found the one reduce job finished time is shorter than the four reduces job. I used tool to search why. Use nmon to monitor the disk read/write operation, I found a phenomena, when one reduce job report finished but the disk write data operation is still sustained for a long time. For four reduce jobs, when job report finished the disk read/write operation is done. In a nutshell, one reduce job finished time add disk write operation time equal to four reduces job finished time. So I think this is the reason why one reduce job time is shorter than the four reduces job, finally, I want to ask why that happened? What is the reduce job finished mark? The information and any attached documents contained in this message may be confidential and/or legally privileged. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, dissemination, or reproduction is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender immediately by return e-mail and destroy all copies of the original message.
