Hi again, Replying to myself because I didn't spot the problem the first time.
On Fri, Apr 11, 2008 at 01:39:17PM +0200, Dejan Muhamedagic wrote: > Hi Yamauchi-san, > [snip] > > There are three big time gaps where crmd was waiting: > > tengine[558]: 2008/04/11_16:27:02 info: send_rsc_command: Initiating action > 7: prmIpPostgreSQLDB_start_0 on dl380g5d > crmd[553]: 2008/04/11_16:27:21 info: handle_shutdown_request: Creating > shutdown request for dl380g5c > tengine[558]: 2008/04/11_16:28:02 WARN: action_timer_callback: Timer popped > (abort_level=1000000, complete=false) > tengine[558]: 2008/04/11_16:28:02 WARN: print_elem: Action missed its > timeout[Action 7]: In-flight (id: prmIpPostgreSQLDB_start_0, loc: dl380g5d, > priority: 0) The action crmd is waiting for was supposed to happen on dl380g5d, but the network is down so dl380g5c is waiting in vain. This is what happened: 1. dl380g5c (the DC) looses connection 2. admin issues a heartbeat stop on dl380g5c 3. crmd, correctly, wants to move resources to dl380g5d before the shutdown 4. after "deadtime" ccm delivers the new connectivity information, i.e. a split brain occurs 5. crmd (actually tengine) is still waiting for the result of the operation it initiated even though it can't hear from that node anymore; first it times out after one minute and then after three minutes when it finally gives up You should file a bugzilla for pacemaker for CRM/tengine component. You can insert the above analysis. BTW, if you were to wait with the shutdown until the "deadtime" passed, then I guess that you would get very different results, i.e. a much faster shutdown. Thanks, Dejan [snip] _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
