Zhe Zhang created MAPREDUCE-6870: ------------------------------------ Summary: Add configuration for MR job to finish when all reducers are complete (even with unfinished mappers) Key: MAPREDUCE-6870 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6870 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 2.6.1 Reporter: Zhe Zhang
Even with MAPREDUCE-5817, there could still be cases where mappers get scheduled before all reducers are complete, but those mappers run for long time, even after all reducers are complete. This could hurt the performance of large MR jobs. In some cases, mappers don't have any materialize-able outcome other than providing intermediate data to reducers. In that case, the job owner should have the config option to finish the job once all reducers are complete. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org