[ https://issues.apache.org/jira/browse/HDFS-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630792#comment-16630792 ]
Xiaoyu Yao commented on HDFS-13916: ----------------------------------- Thanks [~renxunsaky] for reporting and posting the patch. Patch v4 looks pretty good to me. I just have a few minor comments: DistCpSync.java Line 204: we should check with in case !isRdiff() where the source file system might not be webhdfs or hdfs. {code} else if (fs instanceof WebHdfsFileSystem) {code} Line 262: NIT checkstyle (line linger than 80) TestDistCpSync.java Line 73-77: NIT: unrelated formatting change can be avoided. Line 105: same as above, please avoid formatting only change in other places too. Line 163-171: initData()/changeData() refactor is not needed as we have a single cluster and we can always initData with dfs. Line 311/325: NIT: typo: weather -> whether Line 839/878: can we refactor the common part of testSyncSnapshotDiffWithWebHdfs2 and testSyncSnapshotDiffWithWebHdfs3 into a testHelper to reduce duplicated code? > Distcp SnapshotDiff not completely implemented for supporting WebHdfs > --------------------------------------------------------------------- > > Key: HDFS-13916 > URL: https://issues.apache.org/jira/browse/HDFS-13916 > Project: Hadoop HDFS > Issue Type: Improvement > Components: distcp, webhdfs > Affects Versions: 3.0.1, 3.1.1 > Reporter: Xun REN > Assignee: Xun REN > Priority: Major > Labels: easyfix, newbie, patch > Attachments: HDFS-13916.002.patch, HDFS-13916.003.patch, > HDFS-13916.004.patch, HDFS-13916.005.patch, HDFS-13916.patch > > > [~ljain] has worked on the JIRA: > https://issues.apache.org/jira/browse/HDFS-13052 to provide the possibility > to make DistCP of SnapshotDiff with WebHDFSFileSystem. However, in the patch, > there is no modification for the real java class which is used by launching > the command "hadoop distcp ..." > > You can check in the latest version here: > [https://github.com/apache/hadoop/blob/branch-3.1.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java#L96-L100] > In the method "preSyncCheck" of the class "DistCpSync", we still check if the > file system is DFS. > So I propose to change the class DistCpSync in order to take into > consideration what was committed by Lokesh Jain. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org