I've never seen a case where this happens. Is this a theoretical
question or are you seeing this issue?
Alan.
On Jun 15, 2010, at 8:49 AM, Gang Luo wrote:
Hi,
Is it possible the last MapReduce job in the MR plan only loads
something and stores it without any other processing in between? For
example, when visiting some physical operator, we need to end the
current MR operator after embedding the physical operator into MR
operator, and create a new MR operator for later physical operators.
Unfortunately, the following physical operator is a store, the end
of the entire query. In this case, the last MR operator only contain
load and store without any meaningful work in between. This idle
MapReduce job will degrade the performance. Will this happen in Pig?
Thanks,
-Gang
