Gotta have logs. From all 3 nodes mentioned. Only then can we determine if the problem is at the corosync or pacemaker layer - which is the pre-requisit for figuring out what to do next :)
On Fri, Mar 30, 2012 at 1:30 PM, Gregg Stock <[email protected]> wrote: > I had a circuit breaker go out and take two of the 5 nodes in my cluster > down. Now that their back up and running, they are not rejoining the > cluster. > > Here is what I get from crm_mon -1 > > node 1,2 and 3 itchy, scratchy and walter show the following: > ============ > Last updated: Thu Mar 29 19:04:05 2012 > Last change: Thu Mar 29 19:04:03 2012 via cibadmin on walter > Stack: openais > Current DC: walter - partition with quorum > Version: 1.1.6-3.el6-a02c0f19a00c1eb2527ad38f146ebc0834814558 > 5 Nodes configured, 5 expected votes > 9 Resources configured. > ============ > > Online: [ itchy scratchy walter butthead timmy ] > > > On butthead I get > > ============ > Last updated: Thu Mar 29 19:04:24 2012 > Last change: Thu Mar 29 18:42:09 2012 via cibadmin on itchy > Stack: openais > Current DC: NONE > 5 Nodes configured, 5 expected votes > 9 Resources configured. > ============ > > OFFLINE: [ itchy scratchy walter butthead timmy ] > > > On Timmy, I get > > ============ > Last updated: Thu Mar 29 19:04:20 2012 > Last change: > Current DC: NONE > 0 Nodes configured, unknown expected votes > 0 Resources configured. > ============ > > > I don't have anything important running yet. so I can do a full clean up of > everything if needed. > > I also get some weird behavior with timmy. I brought this node up with the > host name as timmy.example.com and I changed the host name to timmy but when > the cluster is offline timmy.example.com shows up as offline. I enter crm > node delete timmy.example.com and it goes away until timmy goes offline > again. > > Thanks, > Gregg Stock > > > > _______________________________________________ > Pacemaker mailing list: [email protected] > http://oss.clusterlabs.org/mailman/listinfo/pacemaker > > Project Home: http://www.clusterlabs.org > Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf > Bugs: http://bugs.clusterlabs.org > _______________________________________________ Pacemaker mailing list: [email protected] http://oss.clusterlabs.org/mailman/listinfo/pacemaker Project Home: http://www.clusterlabs.org Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf Bugs: http://bugs.clusterlabs.org
