[ https://issues.apache.org/jira/browse/HADOOP-13496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816139#comment-15816139 ]
Ted Yu commented on HADOOP-13496: --------------------------------- The patch applies cleanly on latest trunk. bq. which versions does it affect 2.7.3+ bq. for which the patch is targed to trunk and branch-2 > Include file lengths in Mismatch in length error for distcp > ----------------------------------------------------------- > > Key: HADOOP-13496 > URL: https://issues.apache.org/jira/browse/HADOOP-13496 > Project: Hadoop Common > Issue Type: Improvement > Reporter: Ted Yu > Assignee: Ted Yu > Priority: Minor > Labels: distcp > Attachments: HADOOP-13496.v1.patch, HADOOP-13496.v1.patch > > > Currently RetriableFileCopyCommand doesn't show the perceived lengths in > Mismatch in length error: > {code} > 2016-08-12 10:23:14,231 ERROR [LocalJobRunner Map Task Executor #0] > util.RetriableCommand(89): Failure in Retriable command: Copying > hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260- > c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182 > to hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/10.22.9. > 171%2C53952%2C1471022508087.regiongroup-1.1471022510182 > java.io.IOException: Mismatch in length of > source:hdfs://localhost:53941/user/tyu/test-data/dc7c674a-c463-4798-8260-c5d1e3440a4b/WALs/10.22.9.171,53952,1471022508087/10.22.9.171%2C53952%2C1471022508087.regiongroup-1.1471022510182 > and > target:hdfs://localhost:53941/backupUT/backup_1471022580616/WALs/.distcp.tmp.attempt_local344329843_0006_m_000000_0 > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.compareFileLengths(RetriableFileCopyCommand.java:193) > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doCopy(RetriableFileCopyCommand.java:126) > at > org.apache.hadoop.tools.mapred.RetriableFileCopyCommand.doExecute(RetriableFileCopyCommand.java:99) > at > org.apache.hadoop.tools.util.RetriableCommand.execute(RetriableCommand.java:87) > at > org.apache.hadoop.tools.mapred.CopyMapper.copyFileWithRetry(CopyMapper.java:281) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:253) > at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:50) > {code} > It would be helpful to include what's the expected length and what's the real > length. > Thanks to [~yzhangal] for offline discussion. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org