[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738868#comment-14738868 ] Hadoop QA commented on HDFS-8853: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 16m 3s | Findbugs (version ) appears to be broken on HDFS-7285. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 57s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 15s | The applied patch generated 1 release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 58s | The applied patch generated 5 new checkstyle issues (total was 0, now 5). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 29s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 2m 38s | The patch appears to introduce 4 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 19s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 190m 23s | Tests failed in hadoop-hdfs. | | | | 233m 54s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed unit tests | hadoop.hdfs.web.TestWebHDFSOAuth2 | | | hadoop.hdfs.TestWriteStripedFileWithFailure | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12755100/HDFS-8853-HDFS-7285-06.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / f62237b | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/patchReleaseAuditProblems.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12372/console | This message was automatically generated. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-06.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch, > HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738575#comment-14738575 ] Hadoop QA commented on HDFS-8853: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 57s | Findbugs (version ) appears to be broken on HDFS-7285. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 50s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 7s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 15s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 31s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 37s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 2m 37s | The patch appears to introduce 4 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 17s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 0m 22s | Tests failed in hadoop-hdfs. | | | | 43m 10s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed build | hadoop-hdfs | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12755100/HDFS-8853-HDFS-7285-06.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / f62237b | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/patchReleaseAuditProblems.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12371/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12371/console | This message was automatically generated. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-06.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch, > HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737643#comment-14737643 ] Zhe Zhang commented on HDFS-8853: - [~andreina] Wondering if you've had a chance to look at the above. Thanks.. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch, > HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715729#comment-14715729 ] Zhe Zhang commented on HDFS-8853: - [~andreina] This was a meaningful Jenkins run. Could you verify {{TestSafeModeWithStripedFile}}? {{TestWriteStripedFileWithFailure}} fails even without the change. Please see if you also want to address the findbugs warnings. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch, > HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712775#comment-14712775 ] Hadoop QA commented on HDFS-8853: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 27s | Findbugs (version ) appears to be broken on HDFS-7285. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 42s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 16s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 33s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 33s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 2m 38s | The patch appears to introduce 4 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 2s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 175m 55s | Tests failed in hadoop-hdfs. | | | | 217m 26s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed unit tests | hadoop.hdfs.TestWriteStripedFileWithFailure | | | hadoop.hdfs.TestSafeModeWithStripedFile | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12752200/HDFS-8853-HDFS-7285-05.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / 6b6a63b | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/patchReleaseAuditProblems.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12130/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12130/console | This message was automatically generated. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch, > HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712541#comment-14712541 ] Zhe Zhang commented on HDFS-8853: - Thanks for the update [~andreina]. Since there are many failures (likely to be unrelated though), I just triggered Jenkins again. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch, > HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14712520#comment-14712520 ] J.Andreina commented on HDFS-8853: -- Testcase failures are not related to this patch. Please review. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch, > HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14707460#comment-14707460 ] Hadoop QA commented on HDFS-8853: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 32s | Findbugs (version ) appears to be broken on HDFS-7285. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 8m 37s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 11m 9s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 17s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 43s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 57s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 39s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 3m 53s | The patch appears to introduce 5 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 55s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 126m 48s | Tests failed in hadoop-hdfs. | | | | 173m 34s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed unit tests | hadoop.hdfs.server.namenode.ha.TestDNFencing | | | hadoop.hdfs.server.namenode.TestFileTruncate | | Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestFsck | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12751685/HDFS-8853-HDFS-7285-merge-04.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / b57c9a3 | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/patchReleaseAuditProblems.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12071/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12071/console | This message was automatically generated. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch, > HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706542#comment-14706542 ] Vinayakumar B commented on HDFS-8853: - Thanks for the update [~andreina]. Few more nits. 1. {{+boolean validPolicy = false;}}, can move this inside else block itself. 2. {code}+throw new IllegalArgumentException("Policy [ " + ecPolicy.getName() ++ " ] does not match any of the supported policies. " + +"Please select any one of " + ecPolicyNames); + }{code} Just replace this as below {code}+throw new HadoopIllegalArgumentException("Policy '" + ecPolicy.getName() ++ "' does not match any of the supported policies. " + +"Please select any one of " + ecPolicyNames); + }{code} Just to use, {{HadoopIllegalArgumentException}} and making message consistent with command line, "Policy {color:red}[{color} " + ecPolicy.getName() + " {color:red}]{color} " to "Policy {color:green}'{color}" + ecPolicy.getName() + "{color:green}'{color} ". > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch, > HDFS-8853-HDFS-7285-merge-04.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706299#comment-14706299 ] Zhe Zhang commented on HDFS-8853: - Thanks Vinay for reviewing. Agree that passing policy name can be a separate JIRA. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14706296#comment-14706296 ] Vinayakumar B commented on HDFS-8853: - bq. In the long term, it might be better for the client to pass a String to NN instead of the actual policy/schema. [~zhz], I think this could go as a separate Jira, as this involves changing the protocol, changes might be more for this. [~andreina], thanks for the patch. Looks overall good. Few comments, 1. Vaildation should happen only for the user passed policy, not for the system default policy. 2. {{ErasureCodingPolicy[] activePolicies = dir.getFSNamesystem().getErasureCodingPolicies();}}, instead of this, use {{FSDirErasureCodingOp.getErasureCodingPolicies(..);}} directly. Just to avoid one more lock, as already lock will be held. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14705346#comment-14705346 ] Hadoop QA commented on HDFS-8853: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 15m 48s | Findbugs (version ) appears to be broken on HDFS-7285. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 55s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 9s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 15s | The applied patch generated 1 release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 40s | There were no new checkstyle issues. | | {color:red}-1{color} | whitespace | 0m 0s | The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 43s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 3m 33s | The patch appears to introduce 5 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 24s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 182m 53s | Tests failed in hadoop-hdfs. | | | | 226m 59s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed unit tests | hadoop.hdfs.TestLeaseRecovery2 | | | hadoop.hdfs.web.TestWebHDFS | | | hadoop.hdfs.TestWriteStripedFileWithFailure | | | hadoop.hdfs.server.namenode.TestFileTruncate | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12751455/HDFS-8853-HDFS-7285-merge-02.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | HDFS-7285 / b57c9a3 | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/patchReleaseAuditProblems.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/whitespace.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12056/console | This message was automatically generated. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14704587#comment-14704587 ] J.Andreina commented on HDFS-8853: -- Thanks [~zhz] . Sorry for the delay in updating the patch. Please review. bq. In the long term, it might be better for the client to pass a String to NN instead of the actual policy/schema. Here do you mean , to pass only the policy name which creating the zone.? > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch, > HDFS-8853-HDFS-7285-merge-02.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[ https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697873#comment-14697873 ] Zhe Zhang commented on HDFS-8853: - Thanks [~andreina] for the patch. Do you mind rebasing it? I was also thinking about the issue when creating HDFS-8833 patch. In the long term, it might be better for the client to pass a {{String}} to NN instead of the actual policy/schema. > Erasure Coding: Provide ECSchema validation when creating ECZone > > > Key: HDFS-8853 > URL: https://issues.apache.org/jira/browse/HDFS-8853 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Rakesh R >Assignee: J.Andreina > Attachments: HDFS-8853-HDFS-7285-01.patch > > > Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}} > doesn't have any validation that the given {{ecSchema}} is available in > {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists > then will create the ECZone with {{null}} schema. IMHO we could improve this > by doing necessary basic sanity checks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)