[ https://issues.apache.org/jira/browse/HDFS-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13449025#comment-13449025 ]
Hudson commented on HDFS-3054: ------------------------------ Integrated in Hadoop-Mapreduce-trunk-Commit #2709 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2709/]) HDFS-3054. distcp -skipcrccheck has no effect. Contributed by Colin Patrick McCabe. (Revision 1381296) Result = FAILURE todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1381296 Files : * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java > distcp -skipcrccheck has no effect > ---------------------------------- > > Key: HDFS-3054 > URL: https://issues.apache.org/jira/browse/HDFS-3054 > Project: Hadoop HDFS > Issue Type: Bug > Components: tools > Affects Versions: 0.23.2, 2.0.0-alpha, 2.0.1-alpha, 2.2.0-alpha > Reporter: patrick white > Assignee: Colin Patrick McCabe > Fix For: 3.0.0, 2.2.0-alpha > > Attachments: HDFS-3054.002.patch, HDFS-3054.004.patch, hdfs-3054.patch > > > Using distcp with '-skipcrccheck' still seems to cause CRC checksums to > happen. > Ran into this while debugging an issue associated with source and destination > having different blocksizes, and not using the preserve blocksize parameter > (-pb). In both 23.1 and 23.2 builds, trying to bypass the checksum > verification by using the '-skipcrcrcheck' parameter had no effect, the > distcp still failed on checksum errors. > Test scenario to reproduce; > do not use '-pb' and try a distcp from 20.205 (default blksize=128M) to .23 > (default blksize=256M), the distcp fails on checksum errors, which is > expected due to checksum calculation (tiered aggregation of all blks). Trying > the same distcp only providing '-skipcrccheck' still fails with the same > checksum error, it is expected that checksum would now be bypassed and the > distcp would proceed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira