[
https://issues.apache.org/jira/browse/WHIRR-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599561#comment-13599561
]
Paul Baclace edited comment on WHIRR-693 at 3/12/13 2:00 AM:
-------------------------------------------------------------
I'm just now getting back to this work. First I will update the github fork I
made and see how a patch from that looks.
Steve: I agree that supporting "waves" brings up the need for distributed
barrier synchronization, but this is a separable concern at the Whirr
function/plugin level. In case anyone starts down the path of stronger and more
structured semantics here, notice that when "/etc/init.d/foo start" returns, it
is not guaranteed that foo is running; based on de facto semantics, foo could
be ready, still loading, hung, or failed. The strong point of Whirr is that it
does not require stronger semantics like that provided by upstart or osgi or
juju, etc. That makes Whirr flexible, although there is still a big need for
recipe testing.
Ultimately, whether a service dependency is ready to handle a dependee is
application-specific. In that spirit, the best approach is to provide Whirr
function/plugins that can do something like "from master node, connect to mysql
with user U to db D and count the number of rows in table T, retrying until
timeout Tout". Such condition checkers can go a long way, as long as they can
be easily parameterized to use details from the provisioning phase.
was (Author: pbaclace):
I'm just now getting back to this work. First I will update the github fork
I made and see how a patch from that looks.
> Control order of actions with waves of whirr.instance-templates
> ---------------------------------------------------------------
>
> Key: WHIRR-693
> URL: https://issues.apache.org/jira/browse/WHIRR-693
> Project: Whirr
> Issue Type: Bug
> Components: core
> Reporter: Paul Baclace
> Priority: Critical
>
> A cluster can be specified with "waves" by supporting an optional suffix on
> whirr.instance-templates property in order to provide *running* base-level
> services (like ZK or a master-master mysql) to another layer of services
> during the configuration phase. This enables storing cluster-wide information
> in zookeeper or doing db creation beforehand.
> An example of a two "wave" (avoiding "phase" which already has a meaning)
> cluster:
> whirr.instance-templates.0=1 zookeeper, 2 zookeeper, 4 noop
> The .0 templates run like normal during "whirr launch-cluster" and then,
> during the same Whirr run, the .1 template is applied as a modification of
> the same cluster so that new host instances are not allocated:
> whirr.instance-templates.1=1 my-master, 2 my-gateway, 4 my-worker
> In the second wave, instance provisioning is inhibited, the
> instance-templates must have the same number of commas and same sequence of
> leading numbers, and a role is only allowed to appear in one wave.
> Here is another example:
> whirr.instance-templates.0=1 mysql-master+zookeeper, 1
> mysql-master+zookeeper,4 noop
> whirr.instance-templates.1=1 hadoop-namenode+hadoop-jobtracker, 1
> hbase-master+hadoop-secondarynn,4
> hadoop-tasktracker+hadoop-datanode+hbase-regionserver
> In the first wave, the two mysql-masters form a multi-master ensemble which
> keeps state information about the setup in the Whirr process (much like
> ZooKeeperCluster.getHosts(cluster) informs services/hbase of the quorum).
> In the second wave, nodes are not allocated
> (BootstrapClusterAction.doAction() is inhibited if instances already exist),
> but all phases for LaunchClusterCommand, including beforeBootstrap() and
> afterBootstrap() callbacks, are executed. If other
> whirr.cli.command.*Command are run, they would see a combined
> whirr.instance-templates that works like normal.
> Obviously, there would be no reason to limit this to 2 waves, but I do not
> expect more than 10 waves to be useful, so the pattern could enable a suffix
> of "\.[0-9]".
> Instead of WHIRR-221 which aims to specify a single global order for service
> startup, the wave format has the advantage that it relies on the implicit
> synchronization barriers at phases already supported by Whirr, the phases
> still run actions within phases in parallel, and state generated by a wave is
> available to later waves.
> Conceptually, this merely splits the whirr.instance-templates into waves of
> (bootstrap,install,configure,start). If no .[0-9] suffixes are present, then
> Whirr would behave just like normal.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
