[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16072534#comment-16072534 ] Venkat Ranganathan commented on HDFS-9820: -- Falcon needs to work with multiple versions of Hadoop.

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16072514#comment-16072514 ] Andras Bokor commented on HDFS-9820: Facon uses {{setUseDiff}} method. Once they bump up HDFS version

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16070989#comment-16070989 ] Yongjun Zhang commented on HDFS-9820: - Hi [~venkatnrangan], Thanks for the feedback. Would you please

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069222#comment-16069222 ] Venkat Ranganathan commented on HDFS-9820: -- This is an incompatible change in branch-2. This

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606308#comment-15606308 ] Hudson commented on HDFS-9820: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10676 (See

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15606173#comment-15606173 ] Yongjun Zhang commented on HDFS-9820: - Many thanks [~andrew.wang], committed to branch-2. > Improve

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15603580#comment-15603580 ] Andrew Wang commented on HDFS-9820: --- branch-2.002 LGTM, thanks Yongjun > Improve distcp to support

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590514#comment-15590514 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590464#comment-15590464 ] Yongjun Zhang commented on HDFS-9820: - Many thanks again [~andrew.wang], I have committed to trunk.

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590412#comment-15590412 ] Hudson commented on HDFS-9820: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10640 (See

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590280#comment-15590280 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590004#comment-15590004 ] Andrew Wang commented on HDFS-9820: --- One little nit then, this code block: {code} 80private void

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584754#comment-15584754 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15584722#comment-15584722 ] Hudson commented on HDFS-9820: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10629 (See

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583910#comment-15583910 ] Hadoop QA commented on HDFS-9820: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583517#comment-15583517 ] Yongjun Zhang commented on HDFS-9820: - Hi [~andrew.wang], Thanks for asking, I'm actually in the

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583481#comment-15583481 ] Andrew Wang commented on HDFS-9820: --- Hi Yongjun, looks like this was committed to trunk but not the

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15583331#comment-15583331 ] Hudson commented on HDFS-9820: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10624 (See

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15580659#comment-15580659 ] Yongjun Zhang commented on HDFS-9820: - Done real cluster testing. Will be committing this by tomorrow.

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15577466#comment-15577466 ] Yongjun Zhang commented on HDFS-9820: - Many thanks [~andrew.wang]! > Improve distcp to support

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576981#comment-15576981 ] Andrew Wang commented on HDFS-9820: --- Great work Yongjun, I'm +1. New tests look good. I'm okay with doing

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576823#comment-15576823 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15576781#comment-15576781 ] Yongjun Zhang commented on HDFS-9820: - Hi [~andrew.wang], Many thanks again for the review. I uploaded

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15567156#comment-15567156 ] Andrew Wang commented on HDFS-9820: --- Thanks for the replies Yongjun, sounds good. Took another look, some

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552241#comment-15552241 ] Yongjun Zhang commented on HDFS-9820: - Hi [~andrew.wang], As I verbally addressed your question

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15551122#comment-15551122 ] Yongjun Zhang commented on HDFS-9820: - HI [~andrew.wang], Many thanks again for the review. I uploaded

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15544058#comment-15544058 ] Yongjun Zhang commented on HDFS-9820: - Thanks a lot for your review and comments [~andrew.wang]! Good

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543686#comment-15543686 ] Andrew Wang commented on HDFS-9820: --- Hi Yongjun, thanks for sticking with this one. I had a few meta

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543394#comment-15543394 ] Yongjun Zhang commented on HDFS-9820: - Copied from

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15543389#comment-15543389 ] Yongjun Zhang commented on HDFS-9820: - Copied from

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15539978#comment-15539978 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15539938#comment-15539938 ] Yongjun Zhang commented on HDFS-9820: - Per discussion in HDFS-10314, we are going to continue to work

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15248797#comment-15248797 ] Yongjun Zhang commented on HDFS-9820: - Many thanks to [~jingzhao] for the offline discussion. I created

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242150#comment-15242150 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao], Thanks for proposing offline discussion, I was

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242112#comment-15242112 ] Jing Zhao commented on HDFS-9820: - [~yzhangal], I have a very good understanding about HDFS-10263 But

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242030#comment-15242030 ] Yongjun Zhang commented on HDFS-9820: - Many thanks [~jingzhao]. Good discussion! 1. {quote} No. This

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241943#comment-15241943 ] Jing Zhao commented on HDFS-9820: - bq. One small correction: before we do the incremental copy, we create a

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241868#comment-15241868 ] Yongjun Zhang commented on HDFS-9820: - Thanks a lot [~jingzhao]! My thoughts to share: 1. {quote}

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241643#comment-15241643 ] Jing Zhao commented on HDFS-9820: - Let's say we first have snapshot s1 both both source and target (and the

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15241365#comment-15241365 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao], Thanks a lot for your review and comments!

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239955#comment-15239955 ] Jing Zhao commented on HDFS-9820: - Thanks again for working on this, Yongjun. Some comments: # Even without

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238642#comment-15238642 ] Jing Zhao commented on HDFS-9820: - Sure, I will review the patch today/tomorrow. > Improve distcp to

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238330#comment-15238330 ] Hadoop QA commented on HDFS-9820: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15238275#comment-15238275 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao] and [~yufeigu], If you could help taking a look

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236436#comment-15236436 ] Hadoop QA commented on HDFS-9820: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15236386#comment-15236386 ] Yongjun Zhang commented on HDFS-9820: - HI [~jingzhao], I'm uploading rev 003 to use {{-diff "" }} as

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15231139#comment-15231139 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao], About your other comments: {quote} 2.

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15227388#comment-15227388 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao] and [~yufeigu], Thanks much for your review!

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15215551#comment-15215551 ] Yufei Gu commented on HDFS-9820: Hi Yongjun, Thanks a lot for working on it. I have some comments here. 1.

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15214680#comment-15214680 ] Jing Zhao commented on HDFS-9820: - Thanks for working on this, [~yzhangal]! Some comments on the current

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208273#comment-15208273 ] Hadoop QA commented on HDFS-9820: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15208006#comment-15208006 ] Yongjun Zhang commented on HDFS-9820: - Hi [~jingzhao] and [~yufeigu], I just attached patch rev 002. I

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203081#comment-15203081 ] Yongjun Zhang commented on HDFS-9820: - A hopefully improved version is on the way ... > Improve

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15202510#comment-15202510 ] Hadoop QA commented on HDFS-9820: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153220#comment-15153220 ] Yongjun Zhang commented on HDFS-9820: - Hmm, in theory, I think we could make HDFS-4167 to relax the

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152850#comment-15152850 ] Yongjun Zhang commented on HDFS-9820: - Thanks [~jingzhao]! Agree. HDFS-4167 also has the restriction

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152831#comment-15152831 ] Jing Zhao commented on HDFS-9820: - Thanks for creating the jira, [~yzhangal]. I think this jira is not very