On Sat, Sep 10, 2011 at 08:30:59AM -0700, Michael Moon wrote: > >Date: Thu, 8 Sep 2011 18:10:26 +0200 > >From: Lars Ellenberg <[email protected]> > >Subject: Re: [Linux-HA] Primary node restart on Standby node link > failure? > >To: [email protected] > >Message-ID: <[email protected]> > >Content-Type: text/plain; charset=iso-8859-1 > > > >In the log you posted below, there are two urls. > > >Could you be bothered to visit them? > > >Have you? If they unclear, could you point out where, > >and how can we improve the wording so it will be clear? > > Yes, I did visit them and they don't apply at all to this situation. > The server is not heavily loaded. It is the Standby node and has > nothing running on it. And the other link is for a split brain > situation. That is also not the case here. Could you even be bothered > to read and understand my question? > > >Anyways, if your intention was to test node failure, you should test > >node failure, not total cluster communication breakdown aka split-brain, > >which is what you do here. > > > In my test I have disabled all network interfaces on the standby node, > ie, effectively removed it from the network. How is that not > simulating a node failure? > And how is that split brain?
Well, the node did not fail. You only failed all its communication paths. Which is the definition of split brain. Heartbeat processes on both nodes are still up, are not able to communicate to their respective peer, conclude after deadtime that the respective peer is dead, *would* have caused a stonith, if you had it configured, and continue (respectively start/takeover) services. > The standby machine is totally off the network? > you under the impression that when a machine has a hardware failure, > that it can somehow magically still communicate on the network. Is > that one of those magical invisible self powered NIC with an embedded > os? Gotta get me one of those. If the cluster is in doubt about the state of one of its members, it has to make sure (use stonith). Or it has to assume. If the cluster assumes a node is dead, which then later returns (one of the log lines you posted was "dead node returning after partition", where "partition" here is "cluster split brain", some set of nodes which have not been able to communicate with each other), *that* is where the cluster can detect that a split brain happened, that the assumed-to-be-dead node was alive after all. Now services are running on all nodes, some continued to run, some have been started during takeover. The cluster tries to recover from this by: > >The only remotely useful thing heartbeat in haresources mode can > >do to recover from a split brain is to stop everything, everywhere, and > >restart from scratch. > > Again, I am not testing a split brain situation. But yes, you do. Maybe you do not intend to, but you did, according to the logs you posted. > I was asking if I there is a configuration setting that would not have > the Primary Active Node restart if the Standby node fails. In other > words, I want to configure the failover one way one time. Active to > Standby with no failback, without the Active restarting no matter what > happens to the Standby node. Change your test to do a "reboot -nf" or "echo b > /proc/sysrq-trigger", if you like, and see what happens then. > Is the architecture documented anywhere? I noticed there doesn't > appear to be any diagrams in all these docs. Not on linux-ha.org, not > in the User Guide, not in the docs that are part of this old version > 2.1.4. Ever heard the phrase "A picture is worth a thousand words"? > > So Genius Lars, How would one test a node failure? Is that documented > anywhere? Any that watermark on the http://linux-ha.org/FAQ#heavy_load > page makes it unreadable. You want me to point out where to makes > things clearer. Remove that ridiculous watermarking background. Upgrade to pacemaker. > Lose the attitude on your replies. It is not appreciated and make you > look like one of those weeny little nerds holed up in some dark room > somewhere with no friends and who has never been on a date. If you say so. -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. _______________________________________________ Linux-HA mailing list [email protected] http://lists.linux-ha.org/mailman/listinfo/linux-ha See also: http://linux-ha.org/ReportingProblems
