Hi, I'm trying to get to the bottom of a problem I'm seeing with a cluster. At this stage I'm unclear as to whether the issue is with the config or not - the generated error messages seem unclear. So I'm not sure whether I should be staring at the config or the source code at this point, and would appreciate a clue!
I'm running with some of the (live) resources in an unmanaged state whilst testing fail-over with other (non-dependant) resources. The managed resources are a number of OpenVZ virtual machines (each comprising 3 primitives - file-system + OpenVZ VE + SendArp). The filesystems are on LVM volume groups, and the single LVM PV for each volume group resides on a DRBD volume. There are n virtual machines per DRBD volume. I'm running pacemaker 1.0.9.1+hg15626-1 on Debian 6.0. Here are some of the messages (configuration follows at the end of the email): Apr 27 11:06:35 fig crmd: [395]: info: crm_timer_popped: PEngine Recheck Timer (I_PE_CALC) just popped! Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped ] Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: All 2 cluster nodes are eligible to run resources. Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke: Query 985: Requesting the current CIB: S_POLICY_ENGINE Apr 27 11:06:35 fig crmd: [395]: info: do_pe_invoke_callback: Invoking the PE: query=985, ref=pe_calc-dc-1335521195-1437, seq=184, quorate=1 Apr 27 11:06:35 fig pengine: [394]: notice: unpack_config: On loss of CCM Quorum: Ignore Apr 27 11:06:35 fig pengine: [394]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0 Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node fig is online Apr 27 11:06:35 fig pengine: [394]: info: determine_online_status: Node hazel is online Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource essex03-LVM isnt managed Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource calypso-FS isnt managed Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource calypso-VE isnt managed Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation essex03-DRBD:0_monitor_0 found resource essex03-DRBD:0 active in master mode on hazel Apr 27 11:06:35 fig pengine: [394]: info: native_add_running: resource essex03-DRBD:0 isnt managed Apr 27 11:06:35 fig pengine: [394]: notice: unpack_rsc_op: Operation essex02-DRBD:1_monitor_0 found resource essex02-DRBD:1 active in master mode on hazel Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-FS#011(ocf::heartbeat:Filesystem):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-SendArp#011(ocf::heartbeat:SendArp):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: artemis-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-FS#011(ocf::heartbeat:Filesystem):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-SendArp#011(ocf::heartbeat:SendArp):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: athena-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-FS#011(ocf::heartbeat:Filesystem):#011Started hazel (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-SendArp#011(ocf::heartbeat:SendArp):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: calypso-VE#011(ocf::heartbeat:ManageVE):#011Started hazel (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-FS#011(ocf::heartbeat:Filesystem):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-SendArp#011(ocf::heartbeat:SendArp):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: epione-VE#011(ocf::heartbeat:ManageVE):#011Stopped (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex02-LVM#011(ocf::heartbeat:LVM):#011Started hazel Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex03-LVM#011(ocf::heartbeat:LVM):#011Started hazel (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-FS#011(ocf::heartbeat:Filesystem):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-SendArp#011(ocf::heartbeat:SendArp):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essextest-VE#011(ocf::heartbeat:ManageVE):#011Stopped Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: ms-drbd-essex02 Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Masters: [ hazel ] Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Slaves: [ fig ] Apr 27 11:06:35 fig pengine: [394]: notice: clone_print: Master/Slave Set: ms-drbd-essex03 (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: native_print: essex03-DRBD:0#011(ocf::linbit:drbd):#011Master hazel (unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: short_print: Stopped: [ essex03-DRBD:1 ] Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has failed 1 times on hazel Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: ms-drbd-essex03 can fail 999999 more times on hazel before being forced off Apr 27 11:06:35 fig pengine: [394]: info: get_failcount: ms-drbd-essex03 has failed 1 times on hazel Apr 27 11:06:35 fig pengine: [394]: notice: common_apply_stickiness: ms-drbd-essex03 can fail 999999 more times on hazel before being forced off Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex02-DRBD:0 (Slave fig) Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: Promoted 1 instances of a possible 1 to master Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource artemis-VE allocated to 'nowhere': inactive Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource artemis-SendArp cannot run anywhere Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource athena-VE allocated to 'nowhere': inactive Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource athena-SendArp cannot run anywhere Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-DRBD:0 allocated to hazel: master Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-DRBD:1 allocated to 'nowhere': inactive Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged master essex03-DRBD:0 to remain promoted on hazel Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex03-DRBD:0 (Master hazel) Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: Promoted 1 instances of a possible 1 to master Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource essex03-LVM allocated to hazel: active Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource calypso-FS allocated to hazel: active Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource calypso-VE allocated to hazel: active Apr 27 11:06:35 fig pengine: [394]: info: native_color: Unmanaged resource epione-VE allocated to 'nowhere': inactive Apr 27 11:06:35 fig pengine: [394]: info: native_color: Resource epione-SendArp cannot run anywhere Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex02-DRBD:0 (Slave fig) Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex02: Promoted 1 instances of a possible 1 to master Apr 27 11:06:35 fig pengine: [394]: notice: can_be_master: Forcing unmanaged master essex03-DRBD:0 to remain promoted on hazel Apr 27 11:06:35 fig pengine: [394]: info: master_color: Promoting essex03-DRBD:0 (Master hazel) Apr 27 11:06:35 fig pengine: [394]: info: master_color: ms-drbd-essex03: Promoted 1 instances of a possible 1 to master Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (29s) for essex02-DRBD:0 on fig Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (31s) for essex02-DRBD:1 on hazel Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02 Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (29s) for essex02-DRBD:0 on fig Apr 27 11:06:35 fig pengine: [394]: notice: RecurringOp: Start recurring monitor (31s) for essex02-DRBD:1 on hazel Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex02 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03 Apr 27 11:06:35 fig pengine: [394]: ERROR: create_notification_boundaries: Creating boundaries for ms-drbd-essex03 Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start artemis-FS#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource artemis-SendArp#011(Stopped) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource artemis-VE#011(Stopped unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start athena-FS#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource athena-SendArp#011(Stopped) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource athena-VE#011(Stopped unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource calypso-FS#011(Started unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start calypso-SendArp#011(hazel) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource calypso-VE#011(Started unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start epione-FS#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource epione-SendArp#011(Stopped) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource epione-VE#011(Stopped unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Move resource essex02-LVM#011(Started hazel -> fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-LVM#011(Started unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-FS#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-SendArp#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Start essextest-VE#011(fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Promote essex02-DRBD:0#011(Slave -> Master fig) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Demote essex02-DRBD:1#011(Master -> Slave hazel) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-DRBD:0#011(Master unmanaged) Apr 27 11:06:35 fig pengine: [394]: notice: LogActions: Leave resource essex03-DRBD:1#011(Stopped unmanaged) Apr 27 11:06:35 fig crmd: [395]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ] Apr 27 11:06:35 fig crmd: [395]: info: unpack_graph: Unpacked transition 121: 34 actions in 34 synapses Apr 27 11:06:35 fig crmd: [395]: info: do_te_invoke: Processing graph 121 (ref=pe_calc-dc-1335521195-1437) derived from /var/lib/pengine/pe-input-82.bz2 Apr 27 11:06:35 fig crmd: [395]: info: te_pseudo_action: Pseudo action 53 fired and confirmed Apr 27 11:06:35 fig pengine: [394]: info: process_pe_message: Transition 121: PEngine Input stored in: /var/lib/pengine/pe-input-82.bz2 Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action 119: notify essex02-DRBD:0_pre_notify_demote_0 on fig (local) Apr 27 11:06:35 fig crmd: [395]: info: do_lrm_rsc_op: Performing key=119:121:0:3dcf0e40-239f-4eed-9764-9c29003fa305 op=essex02-DRBD:0_notify_0 ) Apr 27 11:06:35 fig lrmd: [392]: info: rsc:essex02-DRBD:0:594: notify Apr 27 11:06:35 fig crmd: [395]: info: te_rsc_command: Initiating action 121: notify essex02-DRBD:1_pre_notify_demote_0 on hazel Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action essex02-DRBD:1_pre_notify_demote_0 (121) confirmed on hazel (rc=0) Apr 27 11:06:36 fig crmd: [395]: info: process_lrm_event: LRM operation essex02-DRBD:0_notify_0 (call=594, rc=0, cib-update=986, confirmed=true) ok Apr 27 11:06:36 fig crmd: [395]: info: match_graph_event: Action essex02-DRBD:0_pre_notify_demote_0 (119) confirmed on fig (rc=0) Apr 27 11:06:36 fig crmd: [395]: info: te_pseudo_action: Pseudo action 54 fired and confirmed Apr 27 11:06:36 fig crmd: [395]: notice: run_graph: ==================================================== Apr 27 11:06:36 fig crmd: [395]: WARN: run_graph: Transition 121 (Complete=4, Pending=0, Fired=0, Skipped=0, Incomplete=30, Source=/var/lib/pengine/pe-input-82.bz2): Terminated Apr 27 11:06:36 fig crmd: [395]: ERROR: te_graph_trigger: Transition failed: terminated Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Graph 121 (34 actions in 34 synapses): batch-limit=30 jobs, network-delay=60000ms Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 0 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 6]: Pending (id: artemis-FS_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 1 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 8]: Pending (id: athena-FS_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 2 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 12]: Pending (id: calypso-SendArp_start_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 14]: Pending (id: calypso-VE_start_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 3 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 15]: Pending (id: epione-FS_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 4 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 10]: Pending (id: calypso-FS_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 5 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 6 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 21]: Pending (id: essextest-FS_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 18]: Pending (id: essex02-LVM_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 7 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 22]: Pending (id: essextest-SendArp_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 23]: Pending (id: essextest-VE_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 8 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 23]: Pending (id: essextest-VE_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 21]: Pending (id: essextest-FS_start_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 9 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 45]: Pending (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 10 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 28]: Pending (id: essex02-DRBD:0_monitor_29000, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 11 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 115]: Pending (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 12 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 116]: Pending (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 13 was confirmed (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 14 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 120]: Pending (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 15 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 16 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 32]: Pending (id: essex02-DRBD:1_monitor_31000, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 17 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 117]: Pending (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 18 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 118]: Pending (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 19 was confirmed (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 20 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 122]: Pending (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 21 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 45]: Pending (id: ms-drbd-essex02_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 22 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 27]: Pending (id: essex02-DRBD:0_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 23 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 24 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 47]: Pending (id: ms-drbd-essex02_pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 115]: Pending (id: essex02-DRBD:0_pre_notify_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 117]: Pending (id: essex02-DRBD:1_pre_notify_promote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 25 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 46]: Pending (id: ms-drbd-essex02_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 48]: Pending (id: ms-drbd-essex02_confirmed-pre_notify_promote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 26 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 50]: Pending (id: ms-drbd-essex02_confirmed-post_notify_promoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 49]: Pending (id: ms-drbd-essex02_post_notify_promoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 116]: Pending (id: essex02-DRBD:0_post_notify_promote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 118]: Pending (id: essex02-DRBD:1_post_notify_promote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 27 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 28 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 31]: Pending (id: essex02-DRBD:1_demote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 51]: Pending (id: ms-drbd-essex02_demote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 29 was confirmed (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 30 was confirmed (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 31 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 52]: Pending (id: ms-drbd-essex02_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 54]: Completed (id: ms-drbd-essex02_confirmed-pre_notify_demote_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 32 is pending (priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 56]: Pending (id: ms-drbd-essex02_confirmed-post_notify_demoted_0, type: pseduo, priority: 1000000) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 55]: Pending (id: ms-drbd-essex02_post_notify_demoted_0, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 120]: Pending (id: essex02-DRBD:0_post_notify_demote_0, loc: fig, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 122]: Pending (id: essex02-DRBD:1_post_notify_demote_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_graph: Synapse 33 is pending (priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: [Action 2]: Pending (id: all_stopped, type: pseduo, priority: 0) Apr 27 11:06:36 fig crmd: [395]: WARN: print_elem: * [Input 17]: Pending (id: essex02-LVM_stop_0, loc: hazel, priority: 0) Apr 27 11:06:36 fig crmd: [395]: info: te_graph_trigger: Transition 121 is now complete Apr 27 11:06:36 fig crmd: [395]: info: notify_crmd: Transition 121 status: done - <null> Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ] Apr 27 11:06:36 fig crmd: [395]: info: do_state_transition: Starting PEngine Recheck Timer The configuration is as follows: node fig node hazel primitive artemis-FS ocf:heartbeat:Filesystem \ params device="/dev/essex02/artemis" directory="/containers/artemis" fstype="ext4" options="relatime,discard" \ op start interval="0" timeout="60s" \ op start interval="0" timeout="60s" primitive artemis-SendArp ocf:heartbeat:SendArp \ params ip="XYZ.XY.126.102" nic="eth0" primitive artemis-VE ocf:heartbeat:ManageVE \ params veid="2501" \ op start interval="0" timeout="75s" \ op start interval="0" timeout="75s" \ meta target-role="Stopped" is-managed="false" primitive athena-FS ocf:heartbeat:Filesystem \ params device="/dev/essex02/athena" directory="/containers/athena" fstype="ext4" options="relatime,discard" \ op start interval="0" timeout="60s" \ op start interval="0" timeout="60s" \ meta is-managed="true" target-role="Started" primitive athena-SendArp ocf:heartbeat:SendArp \ params ip="XYZ.XY.118.23" nic="eth0" primitive athena-VE ocf:heartbeat:ManageVE \ params veid="2504" \ op start interval="0" timeout="75s" \ op start interval="0" timeout="75s" \ meta target-role="Stopped" is-managed="false" primitive calypso-FS ocf:heartbeat:Filesystem \ params device="/dev/essex03/calypso" directory="/containers/calypso" fstype="ext4" options="relatime,discard" \ op start interval="0" timeout="60s" \ op start interval="0" timeout="60s" \ meta target-role="Started" is-managed="false" primitive calypso-SendArp ocf:heartbeat:SendArp \ params ip="XYZ.XY.126.110" nic="eth0" primitive calypso-VE ocf:heartbeat:ManageVE \ params veid="2000" \ op start interval="0" timeout="75s" \ op start interval="0" timeout="75s" \ meta target-role="Stopped" is-managed="false" primitive epione-FS ocf:heartbeat:Filesystem \ params device="/dev/essex02/epione" directory="/containers/epione" fstype="ext4" options="relatime,discard" \ op start interval="0" timeout="60s" \ op start interval="0" timeout="60s" \ meta target-role="Started" primitive epione-SendArp ocf:heartbeat:SendArp \ params ip="XYZ.XY.118.21" nic="eth0" primitive epione-VE ocf:heartbeat:ManageVE \ params veid="2502" \ op start interval="0" timeout="75s" \ op start interval="0" timeout="75s" \ meta target-role="Stopped" is-managed="false" primitive essex02-DRBD ocf:linbit:drbd \ params drbd_resource="essex02" \ op monitor interval="29s" role="Master" \ op monitor interval="31s" role="Slave" \ op start interval="0" timeout="240s" \ op start interval="0" timeout="100s" primitive essex02-LVM ocf:heartbeat:LVM \ params volgrpname="essex02" \ op start interval="0" timeout="240s" \ op start interval="0" timeout="100s" \ meta target-role="Started" is-managed="true" primitive essex03-DRBD ocf:linbit:drbd \ params drbd_resource="essex03" \ op monitor interval="29s" role="Master" \ op monitor interval="31s" role="Slave" \ op start interval="0" timeout="240s" \ meta target-role="Started" is-managed="false" primitive essex03-LVM ocf:heartbeat:LVM \ params volgrpname="essex03" \ op start interval="0" timeout="240s" \ op start interval="0" timeout="100s" \ meta target-role="Started" is-managed="false" primitive essextest-FS ocf:heartbeat:Filesystem \ params device="/dev/essex02/essextest" directory="/containers/essextest" fstype="ext4" options="relatime,discard" \ op start interval="0" timeout="60s" \ op start interval="0" timeout="60s" \ meta target-role="Started" primitive essextest-SendArp ocf:heartbeat:SendArp \ params ip="XYZ.XY.118.25" nic="eth0" \ meta target-role="Started" primitive essextest-VE ocf:heartbeat:ManageVE \ params veid="2599" \ op start interval="0" timeout="75s" \ op start interval="0" timeout="75s" \ meta target-role="Started" ms ms-drbd-essex02 essex02-DRBD \ meta master-max="1" notify="true" globally-unique="false" target-role="Started" is-managed="true" ms ms-drbd-essex03 essex03-DRBD \ meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" globally-unique="false" target-role="Started" is-managed="false" location ms-drbd-essex02-on-fig ms-drbd-essex02 \ rule $id="ms-drbd-essex02-on-fig-rule" $role="master" 100: #uname eq fig location ms-drbd-essex03-on-hazel ms-drbd-essex03 \ rule $id="ms-drbd-essex03-on-hazel-rule" $role="master" 100: #uname eq hazel colocation artemis-FS-with-essex02-LVM inf: artemis-FS essex02-LVM colocation artemis-SendArp-with-artemis-VE inf: artemis-SendArp artemis-VE colocation artemis-VE-with-artemis-FS inf: artemis-VE artemis-FS colocation athena-FS-with-essex02-LVM inf: athena-FS essex02-LVM colocation athena-SendArp-with-athena-VE inf: athena-SendArp athena-VE colocation athena-VE-with-athena-FS inf: athena-VE athena-FS colocation calypso-FS-on-essex03-LVM inf: calypso-FS essex03-LVM colocation calypso-SendArp-with-calypso-VE inf: calypso-SendArp calypso-VE colocation calypso-VE-with-calypso-FS inf: calypso-VE calypso-FS colocation epione-FS-on-essex02-LVM inf: epione-FS essex02-LVM colocation epione-FS-with-essex02-LVM inf: epione-FS essex02-LVM colocation epione-SendArp-with-epione-VE inf: epione-SendArp epione-VE colocation epione-VE-with-epione-FS inf: epione-VE epione-FS colocation essex02-LVM-with-essex02-DRBD-Master inf: essex02-LVM ms-drbd-essex02:Master colocation essex03LVM-on-ms-drbd-essex03 inf: essex03-LVM ms-drbd-essex03:Master colocation essextest-FS-with-essex02-LVM inf: essextest-FS essex02-LVM colocation essextest-SendArp-with-essextest-VE inf: essextest-SendArp essextest-VE colocation essextest-VE-with-essextest-FS inf: essextest-VE essextest-FS order artemis-FS-before-artemis-VE inf: artemis-FS artemis-VE order artemis-VE-before-artemis-SendArp inf: artemis-VE artemis-SendArp order athena-FS-before-athena-VE inf: athena-FS athena-VE order athena-VE-before-athena-SendArp inf: athena-VE athena-SendArp order calypso-FS-before-calypso-VE inf: calypso-FS calypso-VE order calypso-VE-before-calypso-SendArp inf: calypso-VE calypso-SendArp order epione-FS-before-epione-VE inf: epione-FS epione-VE order epione-VE-before-epione-SendArp inf: epione-VE epione-SendArp order essex02-lvm-before-artemis-FS inf: essex02-LVM artemis-FS order essex02-lvm-before-athena-FS inf: essex02-LVM athena-FS order essex02-lvm-before-calypso-FS inf: essex02-LVM calypso-FS order essex02-lvm-before-epione-FS inf: essex02-LVM epione-FS order essex02-lvm-before-essextest-FS inf: essex02-LVM essextest-FS order essextest-FS-before-essextest-VE inf: essextest-FS essextest-VE order essextest-VE-before-essextest-SendArp inf: essextest-VE essextest-SendArp order ms-drbd-essex02-before-lvm inf: ms-drbd-essex02:promote essex02-LVM:start order ms-drbd-essex03-before-lvm inf: ms-drbd-essex03:promote essex03-LVM:start property $id="cib-bootstrap-options" \ dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \ cluster-infrastructure="openais" \ expected-quorum-votes="2" \ no-quorum-policy="ignore" \ stonith-enabled="false" \ last-lrm-refresh="1335487560" # crm configure verify WARNING: artemis-FS: default timeout 20s for stop is smaller than the advised 60 WARNING: artemis-VE: default timeout 20s for stop is smaller than the advised 75 WARNING: athena-FS: default timeout 20s for stop is smaller than the advised 60 WARNING: athena-VE: default timeout 20s for stop is smaller than the advised 75 WARNING: calypso-FS: default timeout 20s for stop is smaller than the advised 60 WARNING: calypso-VE: default timeout 20s for stop is smaller than the advised 75 WARNING: epione-FS: default timeout 20s for stop is smaller than the advised 60 WARNING: epione-VE: default timeout 20s for stop is smaller than the advised 75 WARNING: essex02-LVM: default timeout 20s for stop is smaller than the advised 30 WARNING: essex03-LVM: default timeout 20s for stop is smaller than the advised 30 WARNING: essextest-FS: default timeout 20s for stop is smaller than the advised 60 WARNING: essextest-VE: default timeout 20s for stop is smaller than the advised 75 WARNING: essex02-DRBD: specified timeout 100s for start is smaller than the advised 240 WARNING: essex02-DRBD: default timeout 20s for stop is smaller than the advised 100 WARNING: essex03-DRBD: default timeout 20s for stop is smaller than the advised 100 -- South East Open Source Solutions Limited Registered in England and Wales with company number 06134732. Registered Office: 2 Powell Gardens, Redhill, Surrey, RH1 1TQ VAT number: 900 6633 53 http://seoss.co.uk/ +44-(0)1273-808309 _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org