[
https://issues.apache.org/jira/browse/MAPREDUCE-7068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409763#comment-16409763
]
Jason Lowe commented on MAPREDUCE-7068:
---------------------------------------
Is there a real problem here or a theoretical one? The JIRA started out
stating cleanupWithLogger is smashing exceptions which isn't the case, and the
YarnUncaughtExceptionHandler will log any catastrophic errors that would
prevent the task from running successfully. I'm not seeing an issue here.
> Fix Reduce Exception was overwrited by ReduceTask
> -------------------------------------------------
>
> Key: MAPREDUCE-7068
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7068
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 2.7.1
> Environment: CentOS 7
> Hadoop-2.7.1
> Hive-1.2.1
> Reporter: tartarus
> Priority: Major
> Attachments: MAPREDUCE_7068.patch
>
>
>
> {code:java}
> try {
> //increment processed counter only if skipping feature is enabled
> boolean incrProcCount = SkipBadRecords.getReducerMaxSkipGroups(job)>0 &&
> SkipBadRecords.getAutoIncrReducerProcCount(job);
>
> ReduceValuesIterator<INKEY,INVALUE> values = isSkipping() ?
> new SkippingReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter, umbilical) :
> new ReduceValuesIterator<INKEY,INVALUE>(rIter,
> comparator, keyClass, valueClass,
> job, reporter);
> values.informReduceProgress();
> while (values.more()) {
> reduceInputKeyCounter.increment(1);
> reducer.reduce(values.getKey(), values, collector, reporter);
> if(incrProcCount) {
> reporter.incrCounter(SkipBadRecords.COUNTER_GROUP,
> SkipBadRecords.COUNTER_REDUCE_PROCESSED_GROUPS, 1);
> }
> values.nextKey();
> values.informReduceProgress();
> }
> reducer.close();
> reducer = null;
>
> out.close(reporter);
> out = null;
> } finally {
> IOUtils.cleanupWithLogger(LOG, reducer);
> closeQuietly(out, reporter);
> }
> }
> {code}
> if {color:#d04437}reducer.close();{color} throw Exception ,
> {color:#d04437}reducer = null;{color} will not run, then
> {color:#d04437}IOUtils.cleanupWithLogger(LOG, reducer); {color}
>
> will throw Exception and overwrite the Exception of reducer.close();
> so we should catch it and print log to help targeting issues
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
