On 12/04/2013, at 12:11 PM, pavan tc <pavan...@gmail.com> wrote: > Hi Andrew, > > Thanks much for looking at this. > > > > Then (after about 15 minutes), I see the following: > > There were no logs at all in between? > > Absolutely none in the syslog. Only the regular monitor logs from my resource > agent which continued to report as secondary.
This is very strange, because the thing that caused the I_PE_CALC is a timer that goes off every 15 minutes. Which would seem to imply that there was a transition of some kind about when the failure happened - but somehow it didnt go into the logs. Could you post the complete logs from 14:00 to 14:30? > I also checked /var/log/cluster/corosync.log. The only difference between > this and the ones in syslog are the messages below: > > From /var/log/cluster/corosync.log: > ------------------------------------------------------- > Apr 10 14:12:38 [3391] vsanqa4 crmd: notice: ais_dispatch_message: > Membership 166060: quorum lost > Apr 10 14:12:38 [3386] vsanqa4 cib: notice: crm_update_peer_state: > crm_update_ais_node: Node vsanqa3[1950617772] - state is now lost > Apr 10 14:12:38 [3391] vsanqa4 crmd: notice: crm_update_peer_state: > crm_update_ais_node: Node vsanqa3[1950617772] - state is now lost > Apr 10 14:12:38 [3391] vsanqa4 crmd: info: peer_update_callback: > vsanqa3 is now lost (was member) > Apr 10 14:12:38 corosync [CPG ] chosen downlist: sender r(0) > ip(172.16.68.117) ; members(old:2 left:1) > Apr 10 14:12:38 corosync [MAIN ] Completed service synchronization, ready to > provide service. > > Apr 10 14:12:38 [3386] vsanqa4 cib: info: cib_process_request: > Operation complete: op cib_modify for section nodes (origin=local/crmd/62, > version=0.668.12): OK (rc=0) > Apr 10 14:12:38 [3386] vsanqa4 cib: info: cib_process_request: > Operation complete: op cib_modify for section cib (origin=local/crmd/64, > version=0.668.14): OK (rc=0) > Apr 10 14:12:38 [3391] vsanqa4 crmd: info: crmd_ais_dispatch: > Setting expected votes to 2 > Apr 10 14:12:38 [3386] vsanqa4 cib: info: cib_process_request: > Operation complete: op cib_modify for section crm_config > (origin=local/crmd/66, version=0.668.15): OK (rc=0) > > The first six out of the 10 messages above were seen on syslog too. Adding > them here for context. The last four are the "extra" messages in corosync.log > > Pavan > > > > > > Apr 10 14:26:46 vsanqa4 crmd[3391]: notice: do_state_transition: State > > transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_TIMER_POPPED > > origin=crm_timer_popped ] > > Apr 10 14:26:46 vsanqa4 pengine[3390]: notice: unpack_config: On loss of > > CCM Quorum: Ignore > > Apr 10 14:26:46 vsanqa4 pengine[3390]: notice: LogActions: Promote > > vha-6f92a1f6-969c-4c41-b9ca-7eb6f83ace2e:0#011(Slave -> Master vsanqa4) > > Apr 10 14:26:46 vsanqa4 pengine[3390]: notice: process_pe_message: > > Calculated Transition 3: /var/lib/pacemaker/pengine/pe-input-392.bz2 > > > > Thanks, > > Pavan > > _______________________________________________ > > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > > > Project Home: http://www.clusterlabs.org > > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > > Bugs: http://bugs.clusterlabs.org > > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org > > _______________________________________________ > Pacemaker mailing list: Pacemaker@oss.clusterlabs.org > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org _______________________________________________ Pacemaker mailing list: Pacemaker@oss.clusterlabs.org http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
