[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13202337#comment-13202337 ] Hudson commented on HDFS-2733: -- Integrated in Hadoop-Hdfs-HAbranch-build #71 (See [https://builds.apache.org/job/Hadoop-Hdfs-HAbranch-build/71/]) HDFS-2733. Document HA configuration and CLI. Contributed by Aaron T. Myers. atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1241183 Files : * /hadoop/common/branches/HDFS-1623/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-1623.txt * /hadoop/common/branches/HDFS-1623/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/HDFSHighAvailability.apt.vm * /hadoop/common/branches/HDFS-1623/hadoop-project/src/site/site.xml Document HA configuration and CLI - Key: HDFS-2733 URL: https://issues.apache.org/jira/browse/HDFS-2733 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ha Affects Versions: HA branch (HDFS-1623) Reporter: Eli Collins Assignee: Aaron T. Myers Fix For: HA branch (HDFS-1623) Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html We need to document the configuration changes in HDFS-2231 and the new CLI introduced by HADOOP-7774. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201160#comment-13201160 ] Eli Collins commented on HDFS-2733: --- Looks good. Comments follow. * Might make sense to use machine and host consistently (maybe pick one). * In the option of running two redundant NameNodes paragraph would mention that we now support an Active/Passive configuration with a hot standby since that's the standard HA description for our configuration. * Perhaps be explicit that there are two types of state, persistent and ephemeral state. That all persistent state modifications and some emphemeral (eg block allocation) are logged, with the remaining ephemeral state (block reports) are handled by the Datanodes communicating two both Namenodes. * Might be worth mentioning that we fence both with the fencers but the NNs also fence operations outside the edits log (eg operations from block reports) * In the NFS server section, technically there's no requirement it be an NFS server. Would mention that the availability of the system is limited by the availability of the shared edits dir, therefore in order to remove all single points of failure there needs to be redundancy for the shared edits dir. Specifically, multiple network paths to the storage, and redundancy in the storage itself (disk, network and power). * Perhaps mention that today manual failover is supported and automatic failover is coming? * Can mention explicitly that the NN/SBN layout allows people to re-use their existing 2NN/SBN hardware configuration. * s/any change/change/ * Wrt syncing metadata directories, perhaps worth noting that you only need to sync the edits directory since the shared edits dir only contains edits (people may be confused why the old image they copied there never gets updated) * Should mention that checkHealth currently doesn't do anything Document HA configuration and CLI - Key: HDFS-2733 URL: https://issues.apache.org/jira/browse/HDFS-2733 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ha Affects Versions: HA branch (HDFS-1623) Reporter: Eli Collins Assignee: Aaron T. Myers Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html, HDFSHighAvailability.html We need to document the configuration changes in HDFS-2231 and the new CLI introduced by HADOOP-7774. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[
https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201413#comment-13201413
]
Uma Maheswara Rao G commented on HDFS-2733:
---
Aaron, Thanks a lot for the document. It really looks great.
couple of typos, please take care while committing the patch if there are no
more comments.
1)For *exaple*:
2) to *transiton* to
3) *transitonToActive* and transitionToStandby
Also along with the below lines, will it be good of explaining how Standby Node
is different than other available checkpointing capable nodes today
(BackupNode, CheckpointNode, Secondary NameNode) ?
{noformat}
Note that, in an HA cluster, the Standby NameNode also performs checkpoints of
the namespace state, and thus it is not necessary to run a Secondary NameNode,
CheckpointNode, or BackupNode in an HA cluster.
{noformat}
Document HA configuration and CLI
-
Key: HDFS-2733
URL: https://issues.apache.org/jira/browse/HDFS-2733
Project: Hadoop HDFS
Issue Type: Sub-task
Components: documentation, ha
Affects Versions: HA branch (HDFS-1623)
Reporter: Eli Collins
Assignee: Aaron T. Myers
Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch,
HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html,
HDFSHighAvailability.html, HDFSHighAvailability.html
We need to document the configuration changes in HDFS-2231 and the new CLI
introduced by HADOOP-7774.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201425#comment-13201425 ] Stephen Barlow commented on HDFS-2733: -- • Consider simplifying the first sentence describing each of the 8 configuration parameters, and have it on a separate line from any additional discussion to improve scanability. Short noun phrases are better than imperatives for introducing parameters (e.g., for #3, Similarly to rpc-address above, set the addresses for both NameNodes' HTTP servers to listen on. becomes The addresses for both NameNodes' HTTP servers to listen on.) Also regarding #3, 'exaple' - 'example' • Unless config parameters must be set in a particular order, consider switching to an unordered list (even though I just referenced an item by number) • Wherever 'once' means 'after', use 'after' (reduces ambiguity) Document HA configuration and CLI - Key: HDFS-2733 URL: https://issues.apache.org/jira/browse/HDFS-2733 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ha Affects Versions: HA branch (HDFS-1623) Reporter: Eli Collins Assignee: Aaron T. Myers Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html We need to document the configuration changes in HDFS-2231 and the new CLI introduced by HADOOP-7774. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201513#comment-13201513 ] Todd Lipcon commented on HDFS-2733: --- - In the event of an unplanned event could be worded better - The doc for dfs.federation.nameservices should indicate that, if federation is also being used, that this would be a comma-separated list of nameservice IDs (some of which may be HA and some of which might not be) - I think the example nameservice ID should be something like mycluster or mynameservice rather than ha-nn-uri -- it doubles as the URI but that's a side effect rather than the important bit, I think. - Add a note for dfs.namenode.http-address that https-address should be configured on a secure cluster - For the shell fencing method, should note that the address of the service to be fenced will be inserted as the first argument - This is not yet implemented, and at present will always return success -- if the NN is down it will return non-zero, right? So it still has some basic use. Document HA configuration and CLI - Key: HDFS-2733 URL: https://issues.apache.org/jira/browse/HDFS-2733 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ha Affects Versions: HA branch (HDFS-1623) Reporter: Eli Collins Assignee: Aaron T. Myers Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html We need to document the configuration changes in HDFS-2231 and the new CLI introduced by HADOOP-7774. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2733) Document HA configuration and CLI
[ https://issues.apache.org/jira/browse/HDFS-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13201587#comment-13201587 ] Todd Lipcon commented on HDFS-2733: --- +1, lgtm. Document HA configuration and CLI - Key: HDFS-2733 URL: https://issues.apache.org/jira/browse/HDFS-2733 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation, ha Affects Versions: HA branch (HDFS-1623) Reporter: Eli Collins Assignee: Aaron T. Myers Attachments: HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFS-2733-HDFS-1623.patch, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html, HDFSHighAvailability.html We need to document the configuration changes in HDFS-2231 and the new CLI introduced by HADOOP-7774. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
