Hi, We confirmed movement of no-quorum-policy=freeze in four node constitution.
Of course we understand that quorum control does not act in Heartbeat well. We confirmed the service stop of four nodes in the next procedure. Step1) We start four nodes.(3ACT:1STB) Step2) We send cib.xml. ============ Last updated: Fri Sep 10 14:16:30 2010 Stack: Heartbeat Current DC: srv04 (96faf899-13a6-4550-9d3b-b784f7241d06) - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 4 Nodes configured, unknown expected votes 7 Resources configured. ============ Online: [ srv01 srv02 srv03 srv04 ] Resource Group: Group01 Dummy01 (ocf::heartbeat:Dummy): Started srv01 Dummy01-2 (ocf::heartbeat:Dummy): Started srv01 Resource Group: Group02 Dummy02 (ocf::heartbeat:Dummy): Started srv02 Dummy02-2 (ocf::heartbeat:Dummy): Started srv02 Resource Group: Group03 Dummy03 (ocf::heartbeat:Dummy): Started srv03 Dummy03-2 (ocf::heartbeat:Dummy): Started srv03 Resource Group: grpStonith1 prmStonith1-3 (stonith:external/ssh): Started srv01 Resource Group: grpStonith2 prmStonith2-3 (stonith:external/ssh): Started srv02 Resource Group: grpStonith3 prmStonith3-3 (stonith:external/ssh): Started srv03 Resource Group: grpStonith4 prmStonith4-3 (stonith:external/ssh): Started srv04 Step3) We stop the first node after being stable. [r...@srv02 ~]# crm_mon -1 ============ Last updated: Fri Sep 10 14:17:07 2010 Stack: Heartbeat Current DC: srv04 (96faf899-13a6-4550-9d3b-b784f7241d06) - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 4 Nodes configured, unknown expected votes 7 Resources configured. ============ Online: [ srv02 srv03 srv04 ] OFFLINE: [ srv01 ] Resource Group: Group01 Dummy01 (ocf::heartbeat:Dummy): Started srv04 ---->FO Dummy01-2 (ocf::heartbeat:Dummy): Started srv04 ---->FO Resource Group: Group02 Dummy02 (ocf::heartbeat:Dummy): Started srv02 Dummy02-2 (ocf::heartbeat:Dummy): Started srv02 Resource Group: Group03 Dummy03 (ocf::heartbeat:Dummy): Started srv03 Dummy03-2 (ocf::heartbeat:Dummy): Started srv03 Resource Group: grpStonith1 prmStonith1-3 (stonith:external/ssh): Started srv03 Resource Group: grpStonith2 prmStonith2-3 (stonith:external/ssh): Started srv02 Resource Group: grpStonith3 prmStonith3-3 (stonith:external/ssh): Started srv03 Resource Group: grpStonith4 prmStonith4-3 (stonith:external/ssh): Started srv04 Step4) Furthermore, we stop the next node after being stable. * Because a notice of ccm which does not have Quorum is late, two remaining node nodes move the resource. [r...@srv03 ~]# crm_mon -1 ============ Last updated: Fri Sep 10 14:17:59 2010 Stack: Heartbeat Current DC: srv04 (96faf899-13a6-4550-9d3b-b784f7241d06) - partition with quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 4 Nodes configured, unknown expected votes 7 Resources configured. ============ Online: [ srv03 srv04 ] OFFLINE: [ srv01 srv02 ] Resource Group: Group01 Dummy01 (ocf::heartbeat:Dummy): Started srv04 Dummy01-2 (ocf::heartbeat:Dummy): Started srv04 Resource Group: Group02 Dummy02 (ocf::heartbeat:Dummy): Started srv04 ---->FO Dummy02-2 (ocf::heartbeat:Dummy): Started srv04 ---->FO Resource Group: Group03 Dummy03 (ocf::heartbeat:Dummy): Started srv03 Dummy03-2 (ocf::heartbeat:Dummy): Started srv03 Resource Group: grpStonith1 prmStonith1-3 (stonith:external/ssh): Started srv03 Resource Group: grpStonith2 prmStonith2-3 (stonith:external/ssh): Started srv04 Resource Group: grpStonith3 prmStonith3-3 (stonith:external/ssh): Started srv03 Resource Group: grpStonith4 prmStonith4-3 (stonith:external/ssh): Started srv04 Step5) We stop one node after being more stable. * We stopped it since I became have-quorum=0 of cib. [r...@srv03 ~]# cibadmin -Q | more <cib epoch="102" num_updates="3" admin_epoch="0" validate-with="pacemaker-1.0" crm_feature_set="3.0.1" have-quorum="0" dc-uuid="96faf899-13a6-4550-9d3b-b784f 7241d06"> Step6) Some resources moved to the last node. [r...@srv04 ~]# crm_mon -1 ============ Last updated: Fri Sep 10 14:19:43 2010 Stack: Heartbeat Current DC: srv04 (96faf899-13a6-4550-9d3b-b784f7241d06) - partition WITHOUT quorum Version: 1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b 4 Nodes configured, unknown expected votes 7 Resources configured. ============ Online: [ srv04 ] OFFLINE: [ srv01 srv02 srv03 ] Resource Group: Group01 Dummy01 (ocf::heartbeat:Dummy): Started srv04 Dummy01-2 (ocf::heartbeat:Dummy): Started srv04 Resource Group: Group02 Dummy02 (ocf::heartbeat:Dummy): Started srv04 Dummy02-2 (ocf::heartbeat:Dummy): Started srv04 Resource Group: Group03 Dummy03 (ocf::heartbeat:Dummy): Started srv04 ---->Why FO? Dummy03-2 (ocf::heartbeat:Dummy): Started srv04 ---->Why FO? Resource Group: grpStonith1 prmStonith1-3 (stonith:external/ssh): Started srv04 Resource Group: grpStonith2 prmStonith2-3 (stonith:external/ssh): Started srv04 Resource Group: grpStonith4 prmStonith4-3 (stonith:external/ssh): Started srv04 We thought that the resource that I left in a left node in Step5 did not move last. Because the reason is because it appoints no-quorum-policy=freeze. However, the starting resource seems to move at the time of no-quorum-policy=freeze when I watch a source code. (snip) action_t * custom_action(resource_t *rsc, char *key, const char *task, node_t *on_node, gboolean optional, gboolean save_action, pe_working_set_t *data_set) { action_t *action = NULL; GListPtr possible_matches = NULL; CRM_CHECK(key != NULL, return NULL); CRM_CHECK(task != NULL, return NULL); (snip) } else if(is_set(data_set->flags, pe_flag_have_quorum) == FALSE && data_set->no_quorum_policy == no_quorum_freeze) { crm_debug_3("Check resource is already active"); if(rsc->fns->active(rsc, TRUE) == FALSE) { action->runnable = FALSE; crm_debug("%s\t%s (cancelled : quorum freeze)", action->node->details->uname, action->uuid); } } else { (snip) Is this specifications of right no-quorum-policy=freeze movement? Is there detailed explanation of the no-quorum-policy=freeze movement somewhere? Best Regards, Hideo Yamauchi. _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://developerbugs.linux-foundation.org/enter_bug.cgi?product=Pacemaker