[ 
https://issues.apache.org/jira/browse/HDFS-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16630792#comment-16630792
 ] 

Xiaoyu Yao commented on HDFS-13916:
-----------------------------------

Thanks [~renxunsaky] for reporting and posting the patch. Patch v4 looks pretty 
good to me.

I just have a few minor comments:

DistCpSync.java

Line 204: we should check with in case !isRdiff() where the source file system 
might not be webhdfs or hdfs.

{code} 

else if (fs instanceof WebHdfsFileSystem)

{code}

 

Line 262: NIT checkstyle (line linger than 80)

 

TestDistCpSync.java

Line 73-77: NIT: unrelated formatting change can be avoided.

Line 105: same as above, please avoid formatting only change in other places 
too.

Line 163-171: initData()/changeData() refactor is not needed as we have a 
single cluster and we can always initData with dfs.

Line 311/325: NIT: typo: weather -> whether

Line 839/878: can we refactor the common part of 
testSyncSnapshotDiffWithWebHdfs2 and 

testSyncSnapshotDiffWithWebHdfs3 into a testHelper to reduce duplicated code?

 

> Distcp SnapshotDiff not completely implemented for supporting WebHdfs
> ---------------------------------------------------------------------
>
>                 Key: HDFS-13916
>                 URL: https://issues.apache.org/jira/browse/HDFS-13916
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: distcp, webhdfs
>    Affects Versions: 3.0.1, 3.1.1
>            Reporter: Xun REN
>            Assignee: Xun REN
>            Priority: Major
>              Labels: easyfix, newbie, patch
>         Attachments: HDFS-13916.002.patch, HDFS-13916.003.patch, 
> HDFS-13916.004.patch, HDFS-13916.005.patch, HDFS-13916.patch
>
>
> [~ljain] has worked on the JIRA: 
> https://issues.apache.org/jira/browse/HDFS-13052 to provide the possibility 
> to make DistCP of SnapshotDiff with WebHDFSFileSystem. However, in the patch, 
> there is no modification for the real java class which is used by launching 
> the command "hadoop distcp ..."
>  
> You can check in the latest version here:
> [https://github.com/apache/hadoop/blob/branch-3.1.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java#L96-L100]
> In the method "preSyncCheck" of the class "DistCpSync", we still check if the 
> file system is DFS. 
> So I propose to change the class DistCpSync in order to take into 
> consideration what was committed by Lokesh Jain.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to