[
https://issues.apache.org/jira/browse/HADOOP-12473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14982052#comment-14982052
]
Prabhu Joseph commented on HADOOP-12473:
----------------------------------------
[~liuml07] Hi, I am having a doubt on the below code of
org.apache.hadoop.tools.mapred.CopyMapper
if (ignoreFailures && exception.getCause() instanceof
RetriableFileCopyCommand.CopyReadException
Why we are using &&, if i use hadoop distcp -i , it has ignore any Exception
right. In MRV1 distcp org/apache/hadoop/tools/DistCp$CopyFilesMapper, it was ||
and it ignores any Exception. But in MRV2, it doesn't.
if (0 == failcount || ignoreReadFailures)
> distcp's ignoring failures should be mutually exclusive with the atomic option
> ------------------------------------------------------------------------------
>
> Key: HADOOP-12473
> URL: https://issues.apache.org/jira/browse/HADOOP-12473
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 2.7.1
> Reporter: Mingliang Liu
> Priority: Critical
> Fix For: 2.8.0
>
>
> In {{CopyMapper::handleFailure}}, the mapper handles failure and will ignore
> it if no it's config key is on. Ignoring failures should be mutually
> exclusive with the atomic option otherwise an incomplete dir is eligible for
> commit defeating the purpose.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
