Elukey has uploaded a new change for review. https://gerrit.wikimedia.org/r/278748
Change subject: HDFS Namenode automatic failover support - bug fixes. ...................................................................... HDFS Namenode automatic failover support - bug fixes. Added documentation to the README file, resolved an issue with the zookeeper_hosts variable. Bug: T129838 Change-Id: I6d716e9dd526835b72f76312bb5e95488d7b458c --- M README.md M manifests/hadoop/namenode.pp 2 files changed, 20 insertions(+), 3 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/operations/puppet/cdh refs/changes/48/278748/1 diff --git a/README.md b/README.md index 7b87f00..bf78384 100644 --- a/README.md +++ b/README.md @@ -118,8 +118,11 @@ For detailed documentation, see the [CDH5 High Availability Guide](http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/latest/CDH5-High-Availability-Guide/cdh5hag_hdfs_ha_config.html). -This puppet module only supports Quorum-based HA storage using JournalNodes. -It does not support NFS based HA. +This puppet module supports Quorum-based HA storage using JournalNodes and HDFS Namenode +Automatic failover via Zookeeper. + +Namenode Automatic failover is enabled and configured automatically simply setting +```zookeeper_hosts``` and configuring the JournalNodes. Your JournalNodes will be automatically configured based on the value of ```$cdh::hadoop::journalnode_hosts```. When ```cdh::hadoop``` is included, @@ -221,6 +224,9 @@ sudo service hadoop-yarn-resourcemanager stop sudo service hadoop-hdfs-namenode stop +# If HDFS Automatic Failover is configured for the Master node: +sudo service hadoop-hdfs-zkfc stop + # On your hadoop worker nodes: sudo service hadoop-hdfs-datanode stop sudo service hadoop-yarn-nodemanager stop @@ -237,6 +243,9 @@ sudo service hadoop-hdfs-namenode start sudo service hadoop-yarn-resourcemanager start +# If HDFS Automatic Failover is configured for the Master node: +sudo service hadoop-hdfs-zkfc start + # Now that your primary NameNode is back up, and # JournalNodes have been initialized, bootstrap # your Standby NameNode(s). Run this command diff --git a/manifests/hadoop/namenode.pp b/manifests/hadoop/namenode.pp index bd95758..ea0e75c 100644 --- a/manifests/hadoop/namenode.pp +++ b/manifests/hadoop/namenode.pp @@ -64,7 +64,15 @@ # Create a znode in ZooKeeper inside of which the automatic failover # system stores its data. The command will create a znode in ZooKeeper # and it needs to be executed only when the znode is not present. - $zookeeper_hosts_string = join($::cdh::hadoop::zookeper_hosts, ',') + + # Catch-all if the zookeeper_hosts is not an array. + $zookeeper_hosts_string = inline_template('<%= Array(@zookeeper_hosts).join(",") %>') + + # Zookeeper client required to check znodes. + file { '/usr/lib/zookeeper/bin/zkCli.sh': + ensure => present, + } + exec { 'hadoop-hdfs-zkfc-init': command => '/usr/bin/hdfs zkfc -formatZK', user => 'hdfs', -- To view, visit https://gerrit.wikimedia.org/r/278748 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6d716e9dd526835b72f76312bb5e95488d7b458c Gerrit-PatchSet: 1 Gerrit-Project: operations/puppet/cdh Gerrit-Branch: master Gerrit-Owner: Elukey <ltosc...@wikimedia.org> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
