[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14983266#comment-14983266 ] Xiaobing Zhou commented on HDFS-1477: - Thanks [~arpitagarwal] for review! The next patch will address your comments. > Make NameNode Reconfigurable. > - > > Key: HDFS-1477 > URL: https://issues.apache.org/jira/browse/HDFS-1477 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Affects Versions: 2.7.0 >Reporter: Patrick Kling >Assignee: Xiaobing Zhou > Attachments: HDFS-1477.005.patch, HDFS-1477.2.patch, > HDFS-1477.3.patch, HDFS-1477.4.patch, HDFS-1477.patch > > > Modify NameNode to implement the interface Reconfigurable proposed in > HADOOP-7001. This would allow us to change certain configuration properties > without restarting the name node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[
https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981231#comment-14981231
]
Arpit Agarwal commented on HDFS-1477:
-
Hi [~xiaobingo], thanks for taking this up. My comments on the v5 patch:
# FsNameSystem and DatanodeManager need implement Reconfigurable, since their
Reconfigurable methods are only invoked via NameNode.
# What does getNewConf() do? Looks like it just constructs a new config object
by re-reading the config files? There should have been a Javadoc on the base
class method, perhaps we can add one now.
# Can we just skip implementing reconfiguration via the servlet? A better
approach is to have the daemon re-read its config files like HDFS-6808 (which
you pointed out to me offline :-). Looks like you will need a new RPC call and
support on the client. Also it would be good to log the reconfiguration request
in the hdfs audit log.
# Nitpicks: Fix indentation in DatanodeManager#reconfigurePropertyImpl.
# {{#reconfigurePropertyImpl}} - get {{namesystem.writeLock()}} outside the
{{try}} block.
Other suggestions for separate subtasks.
# The list of reconfigurable properties should not be hard-coded. We can add an
annotation to the reconfigurable properties.
# Add {{ReconfigurableBase}} support for atomic updates, i.e. all or no changes
take effect.
# Documentation.
> Make NameNode Reconfigurable.
> -
>
> Key: HDFS-1477
> URL: https://issues.apache.org/jira/browse/HDFS-1477
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
>Affects Versions: 2.7.0
>Reporter: Patrick Kling
>Assignee: Xiaobing Zhou
> Attachments: HDFS-1477.005.patch, HDFS-1477.2.patch,
> HDFS-1477.3.patch, HDFS-1477.4.patch, HDFS-1477.patch
>
>
> Modify NameNode to implement the interface Reconfigurable proposed in
> HADOOP-7001. This would allow us to change certain configuration properties
> without restarting the name node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[
https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14936244#comment-14936244
]
Hadoop QA commented on HDFS-1477:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 18m 29s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 1s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 23s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 26s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 24s | The applied patch generated
37 new checkstyle issues (total was 687, now 703). |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 38s | The patch appears to introduce 3
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 14s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 109m 17s | Tests failed in hadoop-hdfs. |
| | | 156m 5s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestBackupNode |
| | org.apache.hadoop.hdfs.TestFileConcurrentReader |
| | org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | org.apache.hadoop.hdfs.TestBalancerBandwidth |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12764332/HDFS-1477.005.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 6f335e4 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12747/console |
This message was automatically generated.
> Make NameNode Reconfigurable.
> -
>
> Key: HDFS-1477
> URL: https://issues.apache.org/jira/browse/HDFS-1477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 2.7.0
>Reporter: Patrick Kling
>Assignee: Xiaobing Zhou
> Attachments: HDFS-1477.005.patch, HDFS-1477.2.patch,
> HDFS-1477.3.patch, HDFS-1477.4.patch, HDFS-1477.patch
>
>
> Modify NameNode to implement the interface Reconfigurable proposed in
> HADOOP-7001. This would allow us to change certain configuration properties
> without restarting the name node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791336#comment-14791336 ] Xiaobing Zhou commented on HDFS-1477: - I took this over since there's no progress in last years. > Make NameNode Reconfigurable. > - > > Key: HDFS-1477 > URL: https://issues.apache.org/jira/browse/HDFS-1477 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.23.0 >Reporter: Patrick Kling >Assignee: Xiaobing Zhou > Attachments: HDFS-1477.2.patch, HDFS-1477.3.patch, HDFS-1477.patch > > > Modify NameNode to implement the interface Reconfigurable proposed in > HADOOP-7001. This would allow us to change certain configuration properties > without restarting the name node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791343#comment-14791343 ] Xiaobing Zhou commented on HDFS-1477: - Changed this as a sub task of HDFS-9000. > Make NameNode Reconfigurable. > - > > Key: HDFS-1477 > URL: https://issues.apache.org/jira/browse/HDFS-1477 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 0.23.0 >Reporter: Patrick Kling >Assignee: Xiaobing Zhou > Attachments: HDFS-1477.2.patch, HDFS-1477.3.patch, HDFS-1477.patch > > > Modify NameNode to implement the interface Reconfigurable proposed in > HADOOP-7001. This would allow us to change certain configuration properties > without restarting the name node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-1477) Make NameNode Reconfigurable.
[
https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791602#comment-14791602
]
Hadoop QA commented on HDFS-1477:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 22m 14s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 9m 35s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 58s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 24s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 1m 28s | The applied patch generated
37 new checkstyle issues (total was 689, now 705). |
| {color:red}-1{color} | whitespace | 0m 1s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 27s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with
eclipse:eclipse. |
| {color:red}-1{color} | findbugs | 2m 31s | The patch appears to introduce 3
new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native | 3m 12s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 161m 32s | Tests failed in hadoop-hdfs. |
| | | 214m 5s | |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs |
| Failed unit tests | hadoop.hdfs.server.blockmanagement.TestBlockManager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12756388/HDFS-1477.4.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 0832b38 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/artifact/patchprocess/whitespace.txt
|
| Findbugs warnings |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/12498/console |
This message was automatically generated.
> Make NameNode Reconfigurable.
> -
>
> Key: HDFS-1477
> URL: https://issues.apache.org/jira/browse/HDFS-1477
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 2.7.0
>Reporter: Patrick Kling
>Assignee: Xiaobing Zhou
> Attachments: HDFS-1477.2.patch, HDFS-1477.3.patch, HDFS-1477.4.patch,
> HDFS-1477.patch
>
>
> Modify NameNode to implement the interface Reconfigurable proposed in
> HADOOP-7001. This would allow us to change certain configuration properties
> without restarting the name node.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] Commented: (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12996788#comment-12996788 ] Hadoop QA commented on HDFS-1477: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12466342/HDFS-1477.3.patch against trunk revision 1072023. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.TestFileConcurrentReader -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/184//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/184//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/184//console This message is automatically generated. Make NameNode Reconfigurable. - Key: HDFS-1477 URL: https://issues.apache.org/jira/browse/HDFS-1477 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Patrick Kling Assignee: Patrick Kling Fix For: 0.23.0 Attachments: HDFS-1477.2.patch, HDFS-1477.3.patch, HDFS-1477.patch Modify NameNode to implement the interface Reconfigurable proposed in HADOOP-7001. This would allow us to change certain configuration properties without restarting the name node. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12974021#action_12974021 ] Hadoop QA commented on HDFS-1477: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12466342/HDFS-1477.3.patch against trunk revision 1051669. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.server.namenode.TestStorageRestore org.apache.hadoop.hdfs.TestHDFSTrash -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/18//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/18//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HDFS-Build/18//console This message is automatically generated. Make NameNode Reconfigurable. - Key: HDFS-1477 URL: https://issues.apache.org/jira/browse/HDFS-1477 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Patrick Kling Assignee: Patrick Kling Fix For: 0.23.0 Attachments: HDFS-1477.2.patch, HDFS-1477.3.patch, HDFS-1477.patch Modify NameNode to implement the interface Reconfigurable proposed in HADOOP-7001. This would allow us to change certain configuration properties without restarting the name node. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-1477) Make NameNode Reconfigurable.
[ https://issues.apache.org/jira/browse/HDFS-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12971535#action_12971535 ] Jakob Homan commented on HDFS-1477: --- The addInternalServlet call is used for machine-to-machine servlets that users won't use and are not authenticated (or are authenticated over kerberos). This servlet should use the standard add call so that in a secure system the user will be authenticated. Haven't reviewed the rest of the patch; just wanted to throw that out. Make NameNode Reconfigurable. - Key: HDFS-1477 URL: https://issues.apache.org/jira/browse/HDFS-1477 Project: Hadoop HDFS Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Patrick Kling Assignee: Patrick Kling Fix For: 0.23.0 Attachments: HDFS-1477.2.patch, HDFS-1477.patch Modify NameNode to implement the interface Reconfigurable proposed in HADOOP-7001. This would allow us to change certain configuration properties without restarting the name node. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
