03.09.2013, 08:27, "Andrew Beekhof" <and...@beekhof.net>: > On 02/09/2013, at 5:27 PM, Andrey Groshev <gre...@yandex.ru> wrote: > >> 30.08.2013, 07:18, "Andrew Beekhof" <and...@beekhof.net>: >>> On 29/08/2013, at 7:31 PM, Andrey Groshev <gre...@yandex.ru> wrote: >>>> 29.08.2013, 12:25, "Andrey Groshev" <gre...@yandex.ru>: >>>>> 29.08.2013, 02:55, "Andrew Beekhof" <and...@beekhof.net>: >>>>>> On 28/08/2013, at 5:38 PM, Andrey Groshev <gre...@yandex.ru> wrote: >>>>>>> 28.08.2013, 04:06, "Andrew Beekhof" <and...@beekhof.net>: >>>>>>>> On 27/08/2013, at 1:13 PM, Andrey Groshev <gre...@yandex.ru> wrote: >>>>>>>>> 27.08.2013, 05:39, "Andrew Beekhof" <and...@beekhof.net>: >>>>>>>>>> On 26/08/2013, at 3:09 PM, Andrey Groshev <gre...@yandex.ru> >>>>>>>>>> wrote: >>>>>>>>>>> 26.08.2013, 03:34, "Andrew Beekhof" <and...@beekhof.net>: >>>>>>>>>>>> On 23/08/2013, at 9:39 PM, Andrey Groshev <gre...@yandex.ru> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> Hello, >>>>>>>>>>>>> >>>>>>>>>>>>> Today I try remake my test cluster from cman to corosync2. >>>>>>>>>>>>> I drew attention to the following: >>>>>>>>>>>>> If I reset cluster with cman through cibadmin --erase >>>>>>>>>>>>> --force >>>>>>>>>>>>> In cib is still there exist names of nodes. >>>>>>>>>>>> Yes, the cluster puts back entries for all the nodes it know >>>>>>>>>>>> about automagically. >>>>>>>>>>>>> cibadmin -Ql >>>>>>>>>>>>> ..... >>>>>>>>>>>>> <nodes> >>>>>>>>>>>>> <node id="dev-cluster2-node2.unix.tensor.ru" >>>>>>>>>>>>> uname="dev-cluster2-node2"/> >>>>>>>>>>>>> <node id="dev-cluster2-node4.unix.tensor.ru" >>>>>>>>>>>>> uname="dev-cluster2-node4"/> >>>>>>>>>>>>> <node id="dev-cluster2-node3.unix.tensor.ru" >>>>>>>>>>>>> uname="dev-cluster2-node3"/> >>>>>>>>>>>>> </nodes> >>>>>>>>>>>>> .... >>>>>>>>>>>>> >>>>>>>>>>>>> Even if cman and pacemaker running only one node. >>>>>>>>>>>> I'm assuming all three are configured in cluster.conf? >>>>>>>>>>> Yes, there exist list nodes. >>>>>>>>>>>>> And if I do too on cluster with corosync2 >>>>>>>>>>>>> I see only names of nodes which run corosync and pacemaker. >>>>>>>>>>>> Since you're not included your config, I can only guess that >>>>>>>>>>>> your corosync.conf does not have a nodelist. >>>>>>>>>>>> If it did, you should get the same behaviour. >>>>>>>>>>> I try and expected_node and nodelist. >>>>>>>>>> And it didn't work? What version of pacemaker? >>>>>>>>> It does not work as I expected. >>>>>>>> Thats because you've used IP addresses in the node list. >>>>>>>> ie. >>>>>>>> >>>>>>>> node { >>>>>>>> ring0_addr: 10.76.157.17 >>>>>>>> } >>>>>>>> >>>>>>>> try including the node name as well, eg. >>>>>>>> >>>>>>>> node { >>>>>>>> name: dev-cluster2-node2 >>>>>>>> ring0_addr: 10.76.157.17 >>>>>>>> } >>>>>>> The same thing. >>>>>> I don't know what to say. I tested it here yesterday and it worked >>>>>> as expected. >>>>> I found that the reason that You and I have different results - I did >>>>> not have reverse DNS zone for these nodes. >>>>> I know what it should be, but (PACEMAKER + CMAN) worked without a >>>>> reverse area! >>>> Hasty. Deleted all. Reinstalled. Configured. Not working again. Damn! >>> It would have surprised me... pacemaker 1.1.11 doesn't do any dns lookups >>> - reverse or otherwise. >>> Can you set >>> >>> PCMK_trace_files=corosync.c >>> >>> in your environment and retest? >>> >>> On RHEL6 that means putting the following in /etc/sysconfig/pacemaker >>> export PCMK_trace_files=corosync.c >>> >>> It should produce additional logging[1] that will help diagnose the issue. >>> >>> [1] http://blog.clusterlabs.org/blog/2013/pacemaker-logging/ >> Hello, Andrew. >> >> You are a little misunderstood me. > > No, I understood you fine. > >> I wrote that I rushed to judgment. >> After I did the reverse DNS zone, the cluster behaved correctly. >> BUT after I took apart the cluster dropped configs and restarted on the new >> cluster, >> cluster again don't showed all the nodes in the nodes (only node with >> running pacemaker). >> >> A small portion of the log. Full log >> In which (I thought) there is something interesting. >> >> Aug 30 12:31:11 [9986] dev-cluster2-node4 cib: ( corosync.c:423 ) >> trace: check_message_sanity: Verfied message 4: (dest=<all>:cib, >> from=dev-cluster2-node4:cib.9986, compressed=0, size=1551, total=2143) >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( corosync.c:96 ) >> trace: corosync_node_name: Checking 172793107 vs 0 from >> nodelist.node.0.nodeid >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( ipcc.c:378 ) >> debug: qb_ipcc_disconnect: qb_ipcc_disconnect() >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) >> debug: qb_rb_close: Closing ringbuffer: >> /dev/shm/qb-cmap-request-9616-9989-27-header >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) >> debug: qb_rb_close: Closing ringbuffer: >> /dev/shm/qb-cmap-response-9616-9989-27-header >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: (ringbuffer.c:294 ) >> debug: qb_rb_close: Closing ringbuffer: >> /dev/shm/qb-cmap-event-9616-9989-27-header >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( corosync.c:134 ) >> notice: corosync_node_name: Unable to get node name for nodeid >> 172793107 > > I wonder if you need to be including the nodeid too. ie. > > node { > name: dev-cluster2-node2 > ring0_addr: 10.76.157.17 > nodeid: 2 > } > > I _thought_ that was implicit. > Chrissie: is "nodelist.node.%d.nodeid" always available for corosync2 or only > if explicitly defined in the config? >
I begin from end. "nodelist.node.%d.nodeid" available if defined in /etc/corosync/corosync.conf. If I use only "expected_votes", corosync-cmapctl don't show nodelist. And now... interesting garbage turned out: I try change config as you advised. .... quorum { provider: corosync_votequorum } nodelist { node { ring0_addr: 10.76.157.17 name: dev-cluster2-node2 nodeid: 0 } node { ring0_addr: 10.76.157.18 name: dev-cluster2-node3 nodeid: 1 } node { ring0_addr: 10.76.157.19 name: dev-cluster2-node4 nodeid: 2 } } ...... And! # service pacemaker start Starting Pacemaker Cluster Manager: [FAILED] I see logs. (full log: http://pastebin.com/dW7g6URj) .... snip ... Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:501 ) trace: corosync_initialize_nodelist: Initializing corosync nodelist Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [MAIN ] connection created Sep 03 17:28:16 [22111] dev-cluster2-node4 corosync debug [CMAP ] lib_init_fn: conn=0x7f4de13c9870 Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 0 from nodelist.node.0.nodeid Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 1 from nodelist.node.1.nodeid Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:96 ) trace: corosync_node_name: Checking 2 vs 2 from nodelist.node.2.nodeid Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:104 ) trace: corosync_node_name: Searching for node name for 2 in nodelist.node.2 (null) Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:108 ) trace: corosync_node_name: nodelist.node.2.ring0_addr = 10.76.157.19 Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:120 ) trace: corosync_node_name: nodelist.node.2.name = dev-cluster2-node4 1 Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( corosync.c:525 ) crit: corosync_initialize_nodelist: Nodes 2 and 0 share the same name 'dev-cluster2-node4': shutting down Sep 03 17:28:16 [22140] dev-cluster2-node4 crmd: ( xml.c:2637 ) info: crm_xml_cleanup: Cleaning up memory from libxml2 Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:736 ) debug: qb_ipcs_dispatch_connection_request: HUP conn (22135-22140-10) Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipcs.c:578 ) debug: qb_ipcs_disconnect: qb_ipcs_disconnect(22135-22140-10) state:2 Sep 03 17:28:16 [22135] dev-cluster2-node4 cib: ( ipc.c:341 ) info: crm_client_destroy: Destroying 0 events ..... snip .... Ok, try renumbering and received the following: pacemaker starеed, but nodelist: ..... <nodes> <node id="172793107" uname="dev-cluster2-node4.unix.tensor.ru"/> <node id="3" uname="dev-cluster2-node2.unix.tensor.ru"/> <node id="4" uname="dev-cluster2-node3.unix.tensor.ru"/> <node id="5" uname="dev-cluster2-node4.unix.tensor.ru"/> </nodes> ..... Ok, I send cibadmin --erase --force. ..... <nodes> <node id="4" uname="dev-cluster2-node3.unix.tensor.ru"/> <node id="5" uname="dev-cluster2-node4.unix.tensor.ru"/> <node id="3" uname="dev-cluster2-node2.unix.tensor.ru"/> </nodes> ..... I think something, somewhere is not initialized :) >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( cluster.c:338 ) >> notice: get_node_name: Defaulting to uname -n for the local corosync >> node name >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( attrd.c:651 ) >> debug: attrd_cib_callback: Update 4 for probe_complete=true passed >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] HUP >> conn (9616-9989-27) >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] >> qb_ipcs_disconnect(9616-9989-27) state:2 >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] >> epoll_ctl(del): Bad file descriptor (9) >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [MAIN ] >> cs_ipcs_connection_closed() >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [CMAP ] exit_fn >> for conn=0x7fa96bcb31b0 >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [MAIN ] >> cs_ipcs_connection_destroyed() >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] >> Free'ing ringbuffer: /dev/shm/qb-cmap-response-9616-9989-27-header >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] >> Free'ing ringbuffer: /dev/shm/qb-cmap-event-9616-9989-27-header >> Aug 30 12:31:11 [9615] dev-cluster2-node4 corosync debug [QB ] >> Free'ing ringbuffer: /dev/shm/qb-cmap-request-9616-9989-27-header >> Aug 30 12:31:11 [9989] dev-cluster2-node4 attrd: ( corosync.c:423 ) >> trace: check_message_sanity: Verfied message 1: (dest=<all>:attrd, >> from=dev-cluster2-node4:attrd.9989, compressed=0, size=181, total=773) >> Aug 30 12:31:42 [9984] dev-cluster2-node4 pacemakerd: ( mainloop.c:270 ) >> info: crm_signal_dispatch: Invoking handler for signal 10: User >> defined signal 1 >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: ( ipc.c:307 ) >> info: crm_client_new: Connecting 0x16c98e0 for uid=0 gid=0 pid=10007 >> id=f2f15044-8f76-4ea7-a714-984660619ae7 >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: ( ipc_setup.c:476 ) >> debug: handle_new_connection: IPC credentials authenticated >> (9986-10007-13) >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: ( ipc_shm.c:294 ) >> debug: qb_ipcs_shm_connect: connecting to client [10007] >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: (ringbuffer.c:227 ) >> debug: qb_rb_open_2: shm size:524288; real_size:524288; >> rb->word_size:131072 >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: (ringbuffer.c:227 ) >> debug: qb_rb_open_2: shm size:524288; real_size:524288; >> rb->word_size:131072 >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: (ringbuffer.c:227 ) >> debug: qb_rb_open_2: shm size:524288; real_size:524288; >> rb->word_size:131072 >> Aug 30 12:31:59 [9986] dev-cluster2-node4 cib: ( io.c:579 ) >> debug: activateCibXml: Triggering CIB write for cib_erase op >> Aug 30 12:31:59 [9991] dev-cluster2-node4 crmd: (te_callbacks:122 ) >> debug: te_update_diff: Processing diff (cib_erase): 0.9.3 -> 0.11.1 >> (S_IDLE) >> Aug 30 12:31:59 [9991] dev-cluster2-node4 crmd: ( te_utils.c:423 ) >> info: abort_transition_graph: te_update_diff:126 - Triggered >> transition abort (complete=1, node=, tag=diff, id=(null), magic=NA, >> cib=0.11.1) : Non-status change >>>>>>> # corosync-cmapctl |grep nodelist >>>>>>> nodelist.local_node_pos (u32) = 2 >>>>>>> nodelist.node.0.name (str) = dev-cluster2-node2 >>>>>>> nodelist.node.0.ring0_addr (str) = 10.76.157.17 >>>>>>> nodelist.node.1.name (str) = dev-cluster2-node3 >>>>>>> nodelist.node.1.ring0_addr (str) = 10.76.157.18 >>>>>>> nodelist.node.2.name (str) = dev-cluster2-node4 >>>>>>> nodelist.node.2.ring0_addr (str) = 10.76.157.19 >>>>>>> >>>>>>> # corosync-quorumtool -s >>>>>>> Quorum information >>>>>>> ------------------ >>>>>>> Date: Wed Aug 28 11:29:49 2013 >>>>>>> Quorum provider: corosync_votequorum >>>>>>> Nodes: 1 >>>>>>> Node ID: 172793107 >>>>>>> Ring ID: 52 >>>>>>> Quorate: No >>>>>>> >>>>>>> Votequorum information >>>>>>> ---------------------- >>>>>>> Expected votes: 3 >>>>>>> Highest expected: 3 >>>>>>> Total votes: 1 >>>>>>> Quorum: 2 Activity blocked >>>>>>> Flags: >>>>>>> >>>>>>> Membership information >>>>>>> ---------------------- >>>>>>> Nodeid Votes Name >>>>>>> 172793107 1 dev-cluster2-node4 (local) >>>>>>> >>>>>>> # cibadmin -Q >>>>>>> <cib epoch="25" num_updates="3" admin_epoch="0" >>>>>>> validate-with="pacemaker-1.2" crm_feature_set="3.0.7" >>>>>>> cib-last-written="Wed Aug 28 11:24:06 2013" >>>>>>> update-origin="dev-cluster2-node4" update-client="crmd" have-quorum="0" >>>>>>> dc-uuid="172793107"> >>>>>>> <configuration> >>>>>>> <crm_config> >>>>>>> <cluster_property_set id="cib-bootstrap-options"> >>>>>>> <nvpair id="cib-bootstrap-options-dc-version" >>>>>>> name="dc-version" value="1.1.11-1.el6-4f672bc"/> >>>>>>> <nvpair id="cib-bootstrap-options-cluster-infrastructure" >>>>>>> name="cluster-infrastructure" value="corosync"/> >>>>>>> </cluster_property_set> >>>>>>> </crm_config> >>>>>>> <nodes> >>>>>>> <node id="172793107" uname="dev-cluster2-node4"/> >>>>>>> </nodes> >>>>>>> <resources/> >>>>>>> <constraints/> >>>>>>> </configuration> >>>>>>> <status> >>>>>>> <node_state id="172793107" uname="dev-cluster2-node4" >>>>>>> in_ccm="true" crmd="online" crm-debug-origin="do_state_transition" >>>>>>> join="member" expected="member"> >>>>>>> <lrm id="172793107"> >>>>>>> <lrm_resources/> >>>>>>> </lrm> >>>>>>> <transient_attributes id="172793107"> >>>>>>> <instance_attributes id="status-172793107"> >>>>>>> <nvpair id="status-172793107-probe_complete" >>>>>>> name="probe_complete" value="true"/> >>>>>>> </instance_attributes> >>>>>>> </transient_attributes> >>>>>>> </node_state> >>>>>>> </status> >>>>>>> </cib> >>>>>>>>> I figured out a way get around this, but it would be easier to >>>>>>>>> do if the CIB has worked as a with CMAN. >>>>>>>>> I just do not start the main resource if the attribute is not >>>>>>>>> defined or it is not true. >>>>>>>>> This slightly changes the logic of the cluster. >>>>>>>>> But I'm not sure what the correct behavior. >>>>>>>>> >>>>>>>>> libqb 0.14.4 >>>>>>>>> corosync 2.3.1 >>>>>>>>> pacemaker 1.1.11 >>>>>>>>> >>>>>>>>> All build from source in previews week. >>>>>>>>>>> Now in corosync.conf: >>>>>>>>>>> >>>>>>>>>>> totem { >>>>>>>>>>> version: 2 >>>>>>>>>>> crypto_cipher: none >>>>>>>>>>> crypto_hash: none >>>>>>>>>>> interface { >>>>>>>>>>> ringnumber: 0 >>>>>>>>>>> bindnetaddr: 10.76.157.18 >>>>>>>>>>> mcastaddr: 239.94.1.56 >>>>>>>>>>> mcastport: 5405 >>>>>>>>>>> ttl: 1 >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> logging { >>>>>>>>>>> fileline: off >>>>>>>>>>> to_stderr: no >>>>>>>>>>> to_logfile: yes >>>>>>>>>>> logfile: /var/log/cluster/corosync.log >>>>>>>>>>> to_syslog: yes >>>>>>>>>>> debug: on >>>>>>>>>>> timestamp: on >>>>>>>>>>> logger_subsys { >>>>>>>>>>> subsys: QUORUM >>>>>>>>>>> debug: on >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> quorum { >>>>>>>>>>> provider: corosync_votequorum >>>>>>>>>>> } >>>>>>>>>>> nodelist { >>>>>>>>>>> node { >>>>>>>>>>> ring0_addr: 10.76.157.17 >>>>>>>>>>> } >>>>>>>>>>> node { >>>>>>>>>>> ring0_addr: 10.76.157.18 >>>>>>>>>>> } >>>>>>>>>>> node { >>>>>>>>>>> ring0_addr: 10.76.157.19 >>>>>>>>>>> } >>>>>>>>>>> } >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>>>>>>> >>>>>>>>>>> Project Home: http://www.clusterlabs.org >>>>>>>>>>> Getting started: >>>>>>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>>>>>>> Bugs: http://bugs.clusterlabs.org >>>>>>>>>> , >>>>>>>>>> _______________________________________________ >>>>>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>>>>>> >>>>>>>>>> Project Home: http://www.clusterlabs.org >>>>>>>>>> Getting started: >>>>>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>>>>>> Bugs: http://bugs.clusterlabs.org >>>>>>>>> _______________________________________________ >>>>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>>>>> >>>>>>>>> Project Home: http://www.clusterlabs.org >>>>>>>>> Getting started: >>>>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>>>>> Bugs: http://bugs.clusterlabs.org >>>>>>>> , >>>>>>>> _______________________________________________ >>>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>>>> >>>>>>>> Project Home: http://www.clusterlabs.org >>>>>>>> Getting started: >>>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>>>> Bugs: http://bugs.clusterlabs.org >>>>>>> _______________________________________________ >>>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>>> >>>>>>> Project Home: http://www.clusterlabs.org >>>>>>> Getting started: >>>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>>> Bugs: http://bugs.clusterlabs.org >>>>>> , >>>>>> _______________________________________________ >>>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>>> >>>>>> Project Home: http://www.clusterlabs.org >>>>>> Getting started: >>>>>> http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>>> Bugs: http://bugs.clusterlabs.org >>>>> _______________________________________________ >>>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>>> >>>>> Project Home: http://www.clusterlabs.org >>>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>>> Bugs: http://bugs.clusterlabs.org >>>> _______________________________________________ >>>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>>> >>>> Project Home: http://www.clusterlabs.org >>>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>>> Bugs: http://bugs.clusterlabs.org >>> , >>> _______________________________________________ >>> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >>> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >>> >>> Project Home: http://www.clusterlabs.org >>> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >>> Bugs: http://bugs.clusterlabs.org >> _______________________________________________ >> Pacemaker mailing list: Pacemaker@oss.clusterlabs.org >> http://oss.clusterlabs.org/mailman/listinfo/pacemaker >> >> Project Home: http://www.clusterlabs.org >> Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf >> Bugs: http://bugs.clusterlabs.org > > , > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org