On Sat, Apr 28, 2012 at 5:40 AM, Lars Ellenberg <lars.ellenb...@linbit.com> wrote: > On Fri, Apr 27, 2012 at 11:31:23AM +0100, Tim Small wrote: >> Hi, >> >> I'm trying to get to the bottom of a problem I'm seeing with a cluster. >> At this stage I'm unclear as to whether the issue is with the config or >> not - the generated error messages seem unclear. So I'm not sure >> whether I should be staring at the config or the source code at this >> point, and would appreciate a clue! >> >> I'm running with some of the (live) resources in an unmanaged state >> whilst testing fail-over with other (non-dependant) resources. >> >> The managed resources are a number of OpenVZ virtual machines (each >> comprising 3 primitives - file-system + OpenVZ VE + SendArp). The >> filesystems are on LVM volume groups, and the single LVM PV for each >> volume group resides on a DRBD volume. There are n virtual machines per >> DRBD volume. >> >> I'm running pacemaker 1.0.9.1+hg15626-1 on Debian 6.0. Here are some of >> the messages (configuration follows at the end of the email): > > Upgrading to 1.0.12, or 1.1.7, may get you a little further. > It would not solve the "I need to stop that resource first, but I can > not as it is unmanaged" dependency problem you apparently have here.
There's really not a lot the cluster can do in this situation, there's a 50% chance of getting it wrong no matter what we do. In the most recent versions we now log as loudly as possible (LOG_CRIT) that we cant shutdown because something depends on an unmanaged resource. > I think you simply have some copy'n'paste errors in your constraints, > calypso should be ordered with essex03, not 02. > > May not be the only problem, though. > > BTW, LCMC, respectively the cluster resource and constraint graph view > it presents to you, can help to "just see" this kind of error. > > Some more comments inline. > >> Apr 27 11:06:35 fig crmd: [395]: info: crm_timer_popped: PEngine Recheck >> Timer (I_PE_CALC) just popped! >> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition >> S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED >> origin=crm_timer_popped ] >> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: Progressed to >> state S_POLICY_ENGINE after C_TIMER_POPPED >> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: All 2 cluster >> nodes are eligible to run resources. >> Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke: Query 985: Requesting >> the current CIB: S_POLICY_ENGINE >> Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke_callback: Invoking the >> PE: query=985, ref=pe_calc-dc-1335521195-1437, seq=184, quorate=1 >> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_config: On loss of CCM >> Quorum: Ignore >> Apr 27 11:06:35 fig pengine: [394]: info: unpack_config: Node scores: 'red' >> = -INFINITY, 'yellow' = 0, 'green' = 0 >> Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node fig >> is online >> Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node >> hazel is online >> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource >> essex03-LVM isnt managed >> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource >> calypso-FS isnt managed >> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource >> calypso-VE isnt managed >> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation >> essex03-DRBD:0_monitor_0 found resource essex03-DRBD:0 active in master mode >> on hazel >> Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource >> essex03-DRBD:0 isnt managed >> Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation >> essex02-DRBD:1_monitor_0 found resource essex02-DRBD:1 active in master mode >> on hazel >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> artemis-FS#011(ocf::heartbeat:Filesystem):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> artemis-SendArp#011(ocf::heartbeat:SendArp):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> artemis-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> athena-FS#011(ocf::heartbeat:Filesystem):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> athena-SendArp#011(ocf::heartbeat:SendArp):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> athena-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> calypso-FS#011(ocf::heartbeat:Filesystem):#011Started hazel (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> calypso-SendArp#011(ocf::heartbeat:SendArp):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> calypso-VE#011(ocf::heartbeat:ManageVE):#011Started hazel (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> epione-FS#011(ocf::heartbeat:Filesystem):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> epione-SendArp#011(ocf::heartbeat:SendArp):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> epione-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essex02-LVM#011(ocf::heartbeat:LVM):#011Started hazel >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essex03-LVM#011(ocf::heartbeat:LVM):#011Started hazel (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essextest-FS#011(ocf::heartbeat:Filesystem):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essextest-SendArp#011(ocf::heartbeat:SendArp):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essextest-VE#011(ocf::heartbeat:ManageVE):#011Stopped >> Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: >> ms-drbd-essex02 >> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Masters: [ >> hazel ] >> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Slaves: [ fig ] >> Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: >> ms-drbd-essex03 (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: native_print: >> essex03-DRBD:0#011(ocf::linbit:drbd):#011Master hazel (unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Stopped: [ >> essex03-DRBD:1 ] >> Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has >> failed 1 times on hazel >> Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: >> ms-drbd-essex03 can fail 999999 more times on hazel before being forced off >> Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has >> failed 1 times on hazel >> Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: >> ms-drbd-essex03 can fail 999999 more times on hazel before being forced off >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting >> essex02-DRBD:0 (Slave fig) >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: >> Promoted 1 instances of a possible 1 to master >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> artemis-VE allocated to 'nowhere': inactive >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource >> artemis-SendArp cannot run anywhere >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> athena-VE allocated to 'nowhere': inactive >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource >> athena-SendArp cannot run anywhere >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> essex03-DRBD:0 allocated to hazel: master >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> essex03-DRBD:1 allocated to 'nowhere': inactive >> Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged >> master essex03-DRBD:0 to remain promoted on hazel >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting >> essex03-DRBD:0 (Master hazel) >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: >> Promoted 1 instances of a possible 1 to master >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> essex03-LVM allocated to hazel: active >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> calypso-FS allocated to hazel: active >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> calypso-VE allocated to hazel: active >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource >> epione-VE allocated to 'nowhere': inactive >> Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource >> epione-SendArp cannot run anywhere >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting >> essex02-DRBD:0 (Slave fig) >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: >> Promoted 1 instances of a possible 1 to master >> Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged >> master essex03-DRBD:0 to remain promoted on hazel >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting >> essex03-DRBD:0 (Master hazel) >> Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: >> Promoted 1 instances of a possible 1 to master >> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring >> monitor (29s) for essex02-DRBD:0 on fig >> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring >> monitor (31s) for essex02-DRBD:1 on hazel >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex02 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex02 >> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring >> monitor (29s) for essex02-DRBD:0 on fig >> Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring >> monitor (31s) for essex02-DRBD:1 on hazel >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex02 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex02 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex03 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex03 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex03 >> Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: >> Creating boundaries for ms-drbd-essex03 >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> artemis-FS#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> artemis-SendArp#011(Stopped) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> artemis-VE#011(Stopped unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> athena-FS#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> athena-SendArp#011(Stopped) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> athena-VE#011(Stopped unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> calypso-FS#011(Started unmanaged) > > > Just one of the things that go wrong below: calypso-FS is Started, > but unmanaged. > > >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> calypso-SendArp#011(hazel) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> calypso-VE#011(Started unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> epione-FS#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> epione-SendArp#011(Stopped) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> epione-VE#011(Stopped unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Move resource >> essex02-LVM#011(Started hazel -> fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> essex03-LVM#011(Started unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> essextest-FS#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> essextest-SendArp#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start >> essextest-VE#011(fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Promote >> essex02-DRBD:0#011(Slave -> Master fig) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Demote >> essex02-DRBD:1#011(Master -> Slave hazel) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> essex03-DRBD:0#011(Master unmanaged) >> Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource >> essex03-DRBD:1#011(Stopped unmanaged) >> Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition >> S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS >> cause=C_IPC_MESSAGE origin=handle_response ] >> Apr 27 11:06:35 fig crmd: [395]: info: unpack_graph: Unpacked transition >> 121: 34 actions in 34 synapses >> Apr 27 11:06:35 fig crmd: [395]: info: do_te_invoke: Processing graph 121 >> (ref=pe_calc-dc-1335521195-1437) derived from >> /var/lib/pengine/pe-input-82.bz2 >> Apr 27 11:06:35 fig crmd: [395]: info: te_pseudo_action: Pseudo action 53 >> fired and confirmed >> Apr 27 11:06:35 fig pengine: [394]: info: process_pe_message: Transition >> 121: PEngine Input stored in: /var/lib/pengine/pe-input-82.bz2 >> Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action >> 119: notify essex02-DRBD:0_pre_notify_demote_0 on fig (local) >> Apr 27 11:06:35 fig crmd: [395]: info: do_lrm_rsc_op: Performing >> key=119:121:0:3dcf0e40-239f-4eed-9764-9c29003fa305 >> op=essex02-DRBD:0_notify_0 ) >> Apr 27 11:06:35 fig lrmd: [392]: info: rsc:essex02-DRBD:0:594: notify >> Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action >> 121: notify essex02-DRBD:1_pre_notify_demote_0 on hazel >> Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action >> essex02-DRBD:1_pre_notify_demote_0 (121) confirmed on hazel (rc=0) >> Apr 27 11:06:36 fig crmd: [395]: info: process_lrm_event: LRM operation >> essex02-DRBD:0_notify_0 (call=594, rc=0, cib-update=986, confirmed=true) ok >> Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action >> essex02-DRBD:0_pre_notify_demote_0 (119) confirmed on fig (rc=0) >> Apr 27 11:06:36 fig crmd: [395]: info: te_pseudo_action: Pseudo action 54 >> fired and confirmed >> Apr 27 11:06:36 fig crmd: [395]: notice: run_graph: >> ==================================================== >> Apr 27 11:06:36 fig crmd: [395]: WARN: run_graph: Transition 121 >> (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=30, >> Source=/var/lib/pengine/pe-input-82.bz2): Terminated >> Apr 27 11:06:36 fig crmd: [395]: ERROR: te_graph_trigger: Transition failed: >> terminated >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Graph 121 (34 actions in >> 34 synapses): batch-limit=30 jobs, network-delay=60000ms >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 0 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 6]: Pending >> (id: artemis-FS_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: >> Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 1 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 8]: Pending >> (id: athena-FS_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: >> Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 2 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 12]: Pending >> (id: calypso-SendArp_start_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 14]: >> Pending (id: calypso-VE_start_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 3 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 15]: Pending >> (id: epione-FS_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: >> Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 4 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 17]: Pending >> (id: essex02-LVM_stop_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 10]: >> Pending (id: calypso-FS_stop_0, loc: hazel, priority: 0) > > Below: we want to start essex02-LVM on fig, > so we need to stop it on hazel first. > > Above: to stop essex02-LVM, need to stop calypso-FS first. > > We cannot, as calypso-FS is unmanaged. > > Too bad. > >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 5 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 18]: Pending >> (id: essex02-LVM_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: >> Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: >> Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: >> Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 6 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 21]: Pending >> (id: essextest-FS_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: >> Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 7 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 22]: Pending >> (id: essextest-SendArp_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 23]: >> Pending (id: essextest-VE_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 8 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 23]: Pending >> (id: essextest-VE_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 21]: >> Pending (id: essextest-FS_start_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 9 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 27]: Pending >> (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 45]: >> Pending (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 10 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 28]: Pending >> (id: essex02-DRBD:0_monitor_29000, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: >> Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 11 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 115]: Pending >> (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: >> Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 12 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 116]: Pending >> (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: >> Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 13 was confirmed >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 14 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 120]: Pending >> (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: >> Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 15 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 31]: Pending >> (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: >> Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: >> Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 16 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 32]: Pending >> (id: essex02-DRBD:1_monitor_31000, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: >> Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 17 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 117]: Pending >> (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: >> Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 18 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 118]: Pending >> (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: >> Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 19 was confirmed >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 20 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 122]: Pending >> (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: >> Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 21 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 45]: Pending >> (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: >> Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: >> Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 22 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 46]: Pending >> (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: >> Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 23 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 47]: Pending >> (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: >> Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 24 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 48]: Pending >> (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: >> Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 115]: >> Pending (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 117]: >> Pending (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 25 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 49]: Pending >> (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: >> Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: >> Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 26 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 50]: Pending >> (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, >> priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: >> Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 116]: >> Pending (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 118]: >> Pending (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 27 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 51]: Pending >> (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: >> Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: >> Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 28 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 52]: Pending >> (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: >> Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: >> Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 29 was confirmed >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 30 was confirmed >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 31 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 55]: Pending >> (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: >> Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: >> Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, >> priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 32 is pending >> (priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 56]: Pending >> (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, >> priority: 1000000) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: >> Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: >> 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 120]: >> Pending (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 122]: >> Pending (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 33 is pending >> (priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 2]: Pending >> (id: all_stopped, type: pseduo, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: >> Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) >> Apr 27 11:06:36 fig crmd: [395]: info: te_graph_trigger: Transition 121 is >> now complete >> Apr 27 11:06:36 fig crmd: [395]: info: notify_crmd: Transition 121 status: >> done - <null> >> Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: State transition >> S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL >> origin=notify_crmd ] >> Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: Starting PEngine >> Recheck Timer > >> >> >> The configuration is as follows: >> >> >> node fig >> node hazel >> primitive artemis-FS ocf:heartbeat:Filesystem \ >> params device="/dev/essex02/artemis" >> directory="/containers/artemis" fstype="ext4" options="relatime,discard" \ >> op start interval="0" timeout="60s" \ >> op start interval="0" timeout="60s" >> primitive artemis-SendArp ocf:heartbeat:SendArp \ >> params ip="XYZ.XY.126.102" nic="eth0" >> primitive artemis-VE ocf:heartbeat:ManageVE \ >> params veid="2501" \ >> op start interval="0" timeout="75s" \ >> op start interval="0" timeout="75s" \ >> meta target-role="Stopped" is-managed="false" >> primitive athena-FS ocf:heartbeat:Filesystem \ >> params device="/dev/essex02/athena" >> directory="/containers/athena" fstype="ext4" options="relatime,discard" \ >> op start interval="0" timeout="60s" \ >> op start interval="0" timeout="60s" \ >> meta is-managed="true" target-role="Started" >> primitive athena-SendArp ocf:heartbeat:SendArp \ >> params ip="XYZ.XY.118.23" nic="eth0" >> primitive athena-VE ocf:heartbeat:ManageVE \ >> params veid="2504" \ >> op start interval="0" timeout="75s" \ >> op start interval="0" timeout="75s" \ >> meta target-role="Stopped" is-managed="false" >> primitive calypso-FS ocf:heartbeat:Filesystem \ >> params device="/dev/essex03/calypso" >> directory="/containers/calypso" fstype="ext4" options="relatime,discard" \ >> op start interval="0" timeout="60s" \ >> op start interval="0" timeout="60s" \ >> meta target-role="Started" is-managed="false" >> primitive calypso-SendArp ocf:heartbeat:SendArp \ >> params ip="XYZ.XY.126.110" nic="eth0" >> primitive calypso-VE ocf:heartbeat:ManageVE \ >> params veid="2000" \ >> op start interval="0" timeout="75s" \ >> op start interval="0" timeout="75s" \ >> meta target-role="Stopped" is-managed="false" >> primitive epione-FS ocf:heartbeat:Filesystem \ >> params device="/dev/essex02/epione" >> directory="/containers/epione" fstype="ext4" options="relatime,discard" \ >> op start interval="0" timeout="60s" \ >> op start interval="0" timeout="60s" \ >> meta target-role="Started" >> primitive epione-SendArp ocf:heartbeat:SendArp \ >> params ip="XYZ.XY.118.21" nic="eth0" >> primitive epione-VE ocf:heartbeat:ManageVE \ >> params veid="2502" \ >> op start interval="0" timeout="75s" \ >> op start interval="0" timeout="75s" \ >> meta target-role="Stopped" is-managed="false" >> primitive essex02-DRBD ocf:linbit:drbd \ >> params drbd_resource="essex02" \ >> op monitor interval="29s" role="Master" \ >> op monitor interval="31s" role="Slave" \ >> op start interval="0" timeout="240s" \ >> op start interval="0" timeout="100s" >> primitive essex02-LVM ocf:heartbeat:LVM \ >> params volgrpname="essex02" \ >> op start interval="0" timeout="240s" \ >> op start interval="0" timeout="100s" \ >> meta target-role="Started" is-managed="true" >> primitive essex03-DRBD ocf:linbit:drbd \ >> params drbd_resource="essex03" \ >> op monitor interval="29s" role="Master" \ >> op monitor interval="31s" role="Slave" \ >> op start interval="0" timeout="240s" \ >> meta target-role="Started" is-managed="false" >> primitive essex03-LVM ocf:heartbeat:LVM \ >> params volgrpname="essex03" \ >> op start interval="0" timeout="240s" \ >> op start interval="0" timeout="100s" \ >> meta target-role="Started" is-managed="false" >> primitive essextest-FS ocf:heartbeat:Filesystem \ >> params device="/dev/essex02/essextest" >> directory="/containers/essextest" fstype="ext4" options="relatime,discard" \ >> op start interval="0" timeout="60s" \ >> op start interval="0" timeout="60s" \ >> meta target-role="Started" >> primitive essextest-SendArp ocf:heartbeat:SendArp \ >> params ip="XYZ.XY.118.25" nic="eth0" \ >> meta target-role="Started" >> primitive essextest-VE ocf:heartbeat:ManageVE \ >> params veid="2599" \ >> op start interval="0" timeout="75s" \ >> op start interval="0" timeout="75s" \ >> meta target-role="Started" >> ms ms-drbd-essex02 essex02-DRBD \ >> meta master-max="1" notify="true" globally-unique="false" >> target-role="Started" is-managed="true" >> ms ms-drbd-essex03 essex03-DRBD \ >> meta master-max="1" master-node-max="1" clone-max="2" >> clone-node-max="1" notify="true" globally-unique="false" >> target-role="Started" is-managed="false" >> location ms-drbd-essex02-on-fig ms-drbd-essex02 \ >> rule $id="ms-drbd-essex02-on-fig-rule" $role="master" 100: >> #uname eq fig >> location ms-drbd-essex03-on-hazel ms-drbd-essex03 \ >> rule $id="ms-drbd-essex03-on-hazel-rule" $role="master" 100: >> #uname eq hazel >> colocation artemis-FS-with-essex02-LVM inf: artemis-FS essex02-LVM >> colocation artemis-SendArp-with-artemis-VE inf: artemis-SendArp artemis-VE >> colocation artemis-VE-with-artemis-FS inf: artemis-VE artemis-FS >> colocation athena-FS-with-essex02-LVM inf: athena-FS essex02-LVM >> colocation athena-SendArp-with-athena-VE inf: athena-SendArp athena-VE >> colocation athena-VE-with-athena-FS inf: athena-VE athena-FS > >> colocation calypso-FS-on-essex03-LVM inf: calypso-FS essex03-LVM > > Ok, colo calypso with essex03... but then, why ... > >> colocation calypso-SendArp-with-calypso-VE inf: calypso-SendArp calypso-VE >> colocation calypso-VE-with-calypso-FS inf: calypso-VE calypso-FS >> colocation epione-FS-on-essex02-LVM inf: epione-FS essex02-LVM >> colocation epione-FS-with-essex02-LVM inf: epione-FS essex02-LVM >> colocation epione-SendArp-with-epione-VE inf: epione-SendArp epione-VE >> colocation epione-VE-with-epione-FS inf: epione-VE epione-FS >> colocation essex02-LVM-with-essex02-DRBD-Master inf: essex02-LVM >> ms-drbd-essex02:Master >> colocation essex03LVM-on-ms-drbd-essex03 inf: essex03-LVM >> ms-drbd-essex03:Master >> colocation essextest-FS-with-essex02-LVM inf: essextest-FS essex02-LVM >> colocation essextest-SendArp-with-essextest-VE inf: essextest-SendArp >> essextest-VE >> colocation essextest-VE-with-essextest-FS inf: essextest-VE essextest-FS >> order artemis-FS-before-artemis-VE inf: artemis-FS artemis-VE >> order artemis-VE-before-artemis-SendArp inf: artemis-VE artemis-SendArp >> order athena-FS-before-athena-VE inf: athena-FS athena-VE >> order athena-VE-before-athena-SendArp inf: athena-VE athena-SendArp >> order calypso-FS-before-calypso-VE inf: calypso-FS calypso-VE >> order calypso-VE-before-calypso-SendArp inf: calypso-VE calypso-SendArp >> order epione-FS-before-epione-VE inf: epione-FS epione-VE >> order epione-VE-before-epione-SendArp inf: epione-VE epione-SendArp >> order essex02-lvm-before-artemis-FS inf: essex02-LVM artemis-FS >> order essex02-lvm-before-athena-FS inf: essex02-LVM athena-FS > >> order essex02-lvm-before-calypso-FS inf: essex02-LVM calypso-FS > > Order essex02 with calypso? typo? is this supposed to be essex03? > >> order essex02-lvm-before-epione-FS inf: essex02-LVM epione-FS >> order essex02-lvm-before-essextest-FS inf: essex02-LVM essextest-FS >> order essextest-FS-before-essextest-VE inf: essextest-FS essextest-VE >> order essextest-VE-before-essextest-SendArp inf: essextest-VE >> essextest-SendArp >> order ms-drbd-essex02-before-lvm inf: ms-drbd-essex02:promote >> essex02-LVM:start >> order ms-drbd-essex03-before-lvm inf: ms-drbd-essex03:promote >> essex03-LVM:start >> property $id="cib-bootstrap-options" \ >> dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \ >> cluster-infrastructure="openais" \ >> expected-quorum-votes="2" \ >> no-quorum-policy="ignore" \ >> stonith-enabled="false" \ >> last-lrm-refresh="1335487560" >> >> >> # crm configure verify >> WARNING: artemis-FS: default timeout 20s for stop is smaller than the >> advised 60 >> WARNING: artemis-VE: default timeout 20s for stop is smaller than the >> advised 75 >> WARNING: athena-FS: default timeout 20s for stop is smaller than the >> advised 60 >> WARNING: athena-VE: default timeout 20s for stop is smaller than the >> advised 75 >> WARNING: calypso-FS: default timeout 20s for stop is smaller than the >> advised 60 >> WARNING: calypso-VE: default timeout 20s for stop is smaller than the >> advised 75 >> WARNING: epione-FS: default timeout 20s for stop is smaller than the >> advised 60 >> WARNING: epione-VE: default timeout 20s for stop is smaller than the >> advised 75 >> WARNING: essex02-LVM: default timeout 20s for stop is smaller than the >> advised 30 >> WARNING: essex03-LVM: default timeout 20s for stop is smaller than the >> advised 30 >> WARNING: essextest-FS: default timeout 20s for stop is smaller than the >> advised 60 >> WARNING: essextest-VE: default timeout 20s for stop is smaller than the >> advised 75 >> WARNING: essex02-DRBD: specified timeout 100s for start is smaller than >> the advised 240 >> WARNING: essex02-DRBD: default timeout 20s for stop is smaller than the >> advised 100 >> WARNING: essex03-DRBD: default timeout 20s for stop is smaller than the >> advised 100 > > -- > : Lars Ellenberg > : LINBIT | Your Way to High Availability > : DRBD/HA support and consulting http://www.linbit.com > > DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org