[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738575#comment-14738575
]
Hadoop QA commented on HDFS-8853:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 57s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 50s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 7s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 31s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 37s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 37s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 17s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 0m 22s | Tests failed in hadoop-hdfs. |
| | | 43m 10s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed build | hadoop-hdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12755100/HDFS-8853-HDFS-7285-06.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / f62237b |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12371/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12371/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12371/console |
This message was automatically generated.
> Erasure Coding: Provide ECSchema validation when creating ECZone
>
>
> Key: HDFS-8853
> URL: https://issues.apache.org/jira/browse/HDFS-8853
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: J.Andreina
> Attachments: HDFS-8853-HDFS-7285-01.patch,
> HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-06.patch,
> HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch,
> HDFS-8853-HDFS-7285-merge-04.patch
>
>
> Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
> doesn't have any validation that the given {{ecSchema}} is available in
> {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
> then will create the ECZone with {{null}} schema. IMHO we could improve this
> by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738868#comment-14738868
]
Hadoop QA commented on HDFS-8853:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 3s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 57s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:red}-1{color} | checkstyle | 0m 58s | The applied patch generated 5
new checkstyle issues (total was 0, now 5). |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 29s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 38s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 19s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 190m 23s | Tests failed in hadoop-hdfs. |
| | | 233m 54s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.web.TestWebHDFSOAuth2 |
| | hadoop.hdfs.TestWriteStripedFileWithFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12755100/HDFS-8853-HDFS-7285-06.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / f62237b |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12372/console |
This message was automatically generated.
> Erasure Coding: Provide ECSchema validation when creating ECZone
>
>
> Key: HDFS-8853
> URL: https://issues.apache.org/jira/browse/HDFS-8853
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: J.Andreina
> Attachments: HDFS-8853-HDFS-7285-01.patch,
> HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-06.patch,
> HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch,
> HDFS-8853-HDFS-7285-merge-04.patch
>
>
> Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
> doesn't have any validation that the given {{ecSchema}} is available in
> {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
> then will create the ECZone with {{null}} schema. IMHO we could improve this
> by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14737643#comment-14737643
]
Zhe Zhang commented on HDFS-8853:
-
[~andreina] Wondering if you've had a chance to look at the above. Thanks..
> Erasure Coding: Provide ECSchema validation when creating ECZone
>
>
> Key: HDFS-8853
> URL: https://issues.apache.org/jira/browse/HDFS-8853
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Rakesh R
>Assignee: J.Andreina
> Attachments: HDFS-8853-HDFS-7285-01.patch,
> HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch,
> HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch
>
>
> Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
> doesn't have any validation that the given {{ecSchema}} is available in
> {{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
> then will create the ECZone with {{null}} schema. IMHO we could improve this
> by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712775#comment-14712775
]
Hadoop QA commented on HDFS-8853:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 27s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 41s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 16s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 33s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 38s | The patch appears to introduce 4
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 2s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 175m 55s | Tests failed in hadoop-hdfs. |
| | | 217m 26s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestWriteStripedFileWithFailure |
| | hadoop.hdfs.TestSafeModeWithStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12752200/HDFS-8853-HDFS-7285-05.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / 6b6a63b |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12130/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12130/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12130/console |
This message was automatically generated.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch,
HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715729#comment-14715729
]
Zhe Zhang commented on HDFS-8853:
-
[~andreina] This was a meaningful Jenkins run. Could you verify
{{TestSafeModeWithStripedFile}}? {{TestWriteStripedFileWithFailure}} fails even
without the change. Please see if you also want to address the findbugs
warnings.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch,
HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712541#comment-14712541
]
Zhe Zhang commented on HDFS-8853:
-
Thanks for the update [~andreina]. Since there are many failures (likely to be
unrelated though), I just triggered Jenkins again.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch,
HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712520#comment-14712520
]
J.Andreina commented on HDFS-8853:
--
Testcase failures are not related to this patch. Please review.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-05.patch, HDFS-8853-HDFS-7285-merge-02.patch,
HDFS-8853-HDFS-7285-merge-03.patch, HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706542#comment-14706542
]
Vinayakumar B commented on HDFS-8853:
-
Thanks for the update [~andreina].
Few more nits.
1. {{+boolean validPolicy = false;}}, can move this inside else block
itself.
2. {code}+throw new IllegalArgumentException(Policy [ +
ecPolicy.getName()
++ ] does not match any of the supported policies. +
+Please select any one of + ecPolicyNames);
+ }{code}
Just replace this as below
{code}+throw new HadoopIllegalArgumentException(Policy ' +
ecPolicy.getName()
++ ' does not match any of the supported policies. +
+Please select any one of + ecPolicyNames);
+ }{code}
Just to use, {{HadoopIllegalArgumentException}} and making message consistent
with command line, Policy {color:red}[{color} + ecPolicy.getName() +
{color:red}]{color} to Policy {color:green}'{color} + ecPolicy.getName() +
{color:green}'{color} .
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch,
HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707460#comment-14707460
]
Hadoop QA commented on HDFS-8853:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 32s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 37s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 9s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 17s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 43s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 57s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 39s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 53s | The patch appears to introduce 5
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 55s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 126m 48s | Tests failed in hadoop-hdfs. |
| | | 173m 34s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestDNFencing |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestFsck |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12751685/HDFS-8853-HDFS-7285-merge-04.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / b57c9a3 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12071/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12071/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12071/console |
This message was automatically generated.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch,
HDFS-8853-HDFS-7285-merge-04.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706296#comment-14706296
]
Vinayakumar B commented on HDFS-8853:
-
bq. In the long term, it might be better for the client to pass a String to NN
instead of the actual policy/schema.
[~zhz], I think this could go as a separate Jira, as this involves changing the
protocol, changes might be more for this.
[~andreina], thanks for the patch.
Looks overall good.
Few comments,
1. Vaildation should happen only for the user passed policy, not for the system
default policy.
2. {{ErasureCodingPolicy[] activePolicies =
dir.getFSNamesystem().getErasureCodingPolicies();}}, instead of this, use
{{FSDirErasureCodingOp.getErasureCodingPolicies(..);}} directly. Just to avoid
one more lock, as already lock will be held.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706299#comment-14706299
]
Zhe Zhang commented on HDFS-8853:
-
Thanks Vinay for reviewing. Agree that passing policy name can be a separate
JIRA.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch, HDFS-8853-HDFS-7285-merge-03.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704587#comment-14704587
]
J.Andreina commented on HDFS-8853:
--
Thanks [~zhz] .
Sorry for the delay in updating the patch. Please review.
bq. In the long term, it might be better for the client to pass a String to NN
instead of the actual policy/schema.
Here do you mean , to pass only the policy name which creating the zone.?
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705346#comment-14705346
]
Hadoop QA commented on HDFS-8853:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 15m 48s | Findbugs (version ) appears to
be broken on HDFS-7285. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 55s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 9s | There were no new javadoc
warning messages. |
| {color:red}-1{color} | release audit | 0m 15s | The applied patch generated
1 release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 40s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 2 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 43s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 3m 33s | The patch appears to introduce 5
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 24s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 182m 53s | Tests failed in hadoop-hdfs. |
| | | 226m 59s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.TestLeaseRecovery2 |
| | hadoop.hdfs.web.TestWebHDFS |
| | hadoop.hdfs.TestWriteStripedFileWithFailure |
| | hadoop.hdfs.server.namenode.TestFileTruncate |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12751455/HDFS-8853-HDFS-7285-merge-02.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | HDFS-7285 / b57c9a3 |
| Release Audit |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/patchReleaseAuditProblems.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12056/console |
This message was automatically generated.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch,
HDFS-8853-HDFS-7285-merge-02.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-8853) Erasure Coding: Provide ECSchema validation when creating ECZone
[
https://issues.apache.org/jira/browse/HDFS-8853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697873#comment-14697873
]
Zhe Zhang commented on HDFS-8853:
-
Thanks [~andreina] for the patch. Do you mind rebasing it?
I was also thinking about the issue when creating HDFS-8833 patch. In the long
term, it might be better for the client to pass a {{String}} to NN instead of
the actual policy/schema.
Erasure Coding: Provide ECSchema validation when creating ECZone
Key: HDFS-8853
URL: https://issues.apache.org/jira/browse/HDFS-8853
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: Rakesh R
Assignee: J.Andreina
Attachments: HDFS-8853-HDFS-7285-01.patch
Presently the {{DFS#createErasureCodingZone(path, ecSchema, cellSize)}}
doesn't have any validation that the given {{ecSchema}} is available in
{{ErasureCodingSchemaManager#activeSchemas}} list. Now, if it doesn't exists
then will create the ECZone with {{null}} schema. IMHO we could improve this
by doing necessary basic sanity checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
