[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994595#comment-14994595 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1371 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994726#comment-14994726 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #638 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994755#comment-14994755 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2578 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994587#comment-14994587 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #648 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994974#comment-14994974 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2518 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14995014#comment-14995014 ] Hudson commented on HDFS-9236: -- ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #579 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14994439#comment-14994439 ] Hudson commented on HDFS-9236: -- FAILURE: Integrated in Hadoop-trunk-Commit #8769 (See

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992070#comment-14992070 ] Yongjun Zhang commented on HDFS-9236: - Seems jenkins was not triggered, I did one here

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992349#comment-14992349 ] Hadoop QA commented on HDFS-9236: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992724#comment-14992724 ] Tony Wu commented on HDFS-9236: --- Looked at the failed tests and none are related to block recovery. Also

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990903#comment-14990903 ] Walter Su commented on HDFS-9236: - The logic looks good to me. Thanks [~twu] for updating and [~yzhangal]

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990945#comment-14990945 ] Yongjun Zhang commented on HDFS-9236: - Thanks [~twu] for the new rev and [~walter.k.su] for the

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990215#comment-14990215 ] Tony Wu commented on HDFS-9236: --- Thanks a lot [~yzhangal] for your comments. I incorporated them into the new

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988642#comment-14988642 ] Yongjun Zhang commented on HDFS-9236: - HI [~twu], Thanks for the new rev. Some nits. a. Suggest to

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14988859#comment-14988859 ] Hadoop QA commented on HDFS-9236: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989045#comment-14989045 ] Yongjun Zhang commented on HDFS-9236: - Thanks [~twu] for the offline discussion. Consolidating the

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983869#comment-14983869 ] Yongjun Zhang commented on HDFS-9236: - Thanks [~walter.k.su], that makes sense. > Missing sanity

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14984040#comment-14984040 ] Tony Wu commented on HDFS-9236: --- Thanks [~walter.k.su] and [~yzhangal] for your comments. I'll post a new

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982020#comment-14982020 ] Walter Su commented on HDFS-9236: - If a buggy DN does return RUR without throwing

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982006#comment-14982006 ] Zhe Zhang commented on HDFS-9236: - bq. If a DN has a RUR, it will return RecoveryInProgressException.

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983718#comment-14983718 ] Walter Su commented on HDFS-9236: - I agree with [~zhz] that a buggy DN could cause this issue. And I agree

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983731#comment-14983731 ] Walter Su commented on HDFS-9236: - {{syncBlock}} already has an assumption that there's no RUR in syncList.

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983828#comment-14983828 ] Yongjun Zhang commented on HDFS-9236: - Hi [~walter.k.su], Thanks for the comments. Agree that

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983844#comment-14983844 ] Walter Su commented on HDFS-9236: - I mean RUR shouldn't be put in syncList. > Missing sanity check for

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982958#comment-14982958 ] Tony Wu commented on HDFS-9236: --- Thanks a lot for [~walter.k.su] and [~zhz]'s comments! [~walter.k.su], DN

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980912#comment-14980912 ] Tony Wu commented on HDFS-9236: --- [~yzhangal] Thanks a lot for looking at the patch. > Missing sanity check

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980907#comment-14980907 ] Yongjun Zhang commented on HDFS-9236: - Sorry for the delay [~e90tony]. I did a review and I'm +1 on rev

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981071#comment-14981071 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], I believe HDFS-9255 has moved block recovery related code to

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981025#comment-14981025 ] Yongjun Zhang commented on HDFS-9236: - Sorry [~twu], the patch no longer applies because of other

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981330#comment-14981330 ] Mingliang Liu commented on HDFS-9236: - The latest patch looks good to me overall. One minor comment: is

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981248#comment-14981248 ] Yongjun Zhang commented on HDFS-9236: - Thanks [~twu], +1 on rev4 pending jenkins. > Missing sanity

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981458#comment-14981458 ] Mingliang Liu commented on HDFS-9236: - Sorry for the confusion. By "assert expected exception thrown

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981419#comment-14981419 ] Tony Wu commented on HDFS-9236: --- Hi [~liuml07], Thanks a lot for your comment. I debated about having an

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981644#comment-14981644 ] Hadoop QA commented on HDFS-9236: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981585#comment-14981585 ] Tony Wu commented on HDFS-9236: --- Thanks for clarifying. I'll post a updated patch shortly. > Missing sanity

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981604#comment-14981604 ] Mingliang Liu commented on HDFS-9236: - +1 (non-binding) pending on Jenkins. > Missing sanity check for

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981685#comment-14981685 ] Walter Su commented on HDFS-9236: - Please hold on the patch. I doubt if it can happen in real case. If a DN

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967513#comment-14967513 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], Could you take another look at the updated patch? Thanks,

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961543#comment-14961543 ] Tony Wu commented on HDFS-9236: --- checksyle and pre-patch error are not related to this patch. > Missing

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959157#comment-14959157 ] Yongjun Zhang commented on HDFS-9236: - Hi [~twu], Thanks for reporting the finding out the cause of

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959008#comment-14959008 ] Tony Wu commented on HDFS-9236: --- Thanks to [~yzhangal] for offline review and valuable comments! In summary:

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959676#comment-14959676 ] Hadoop QA commented on HDFS-9236: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959255#comment-14959255 ] Hadoop QA commented on HDFS-9236: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959253#comment-14959253 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], Thanks a lot for looking at the patch. Regarding your

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959417#comment-14959417 ] Yongjun Zhang commented on HDFS-9236: - Hi [~twu], Thanks for the updated rev 3 which looks reasonable

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955280#comment-14955280 ] Tony Wu commented on HDFS-9236: --- The path does: * Add replica length check in syncBlock() so DN reports error

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955665#comment-14955665 ] Hadoop QA commented on HDFS-9236: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955818#comment-14955818 ] Tony Wu commented on HDFS-9236: --- All tests pass when manually run on OSX and Linux (CentOS 6.4) with latest