On Thu, 2021-02-11 at 13:59 +0100, Ulrich Windl wrote: > Hi! > > After that problem I see this in crm_mon output: > Failed Fencing Actions: > * reboot of h16 failed: delegate=h18, client=pacemaker- > controld.9087, > origin=h18, last-failed='2021-02-09 14:50:18 +01:00' > > Is there a way to clean that up?
stonith_admin --cleanup -H h16 (or '*') or equivalent in higher-level tool > BTW: h16 had been booted today and still this message is there. Yes, that's a feature. :) As long as any node remains up, they will sync history with each other. That ensures the view is the same regardless of what node you run the command on. > Regards, > Ulrich > > > > > Ulrich Windl schrieb am 09.02.2021 um 16:32 in Nachricht > > > > <6022AB1C.645 : > > 161 : > 60728>: > > > > > Klaus Wenninger <[email protected]> schrieb am 09.02.2021 > > > > > um 16:12 in > > > > Nachricht <[email protected]>: > > > On 2/9/21 3:10 PM, Ulrich Windl wrote: > > > > > > > "Ulrich Windl" <[email protected]> > > > > > > > schrieb am > > 09.02.2021 > > > > um > > > > 15:00 in Nachricht < > > > > [email protected]>: > > > > > Hi! > > > > > > > > > > I had made a mistake, leading to node h16 to be fenced. After > > > > > recovery > > (h16 > > > > > had re‑joined the cluster) I had stopped the node, > > > > > reconfigured the > > network, > > > > > then started the node again. > > > > > Then I did the same thing (not the unwanted fencing) with > > > > > h18. When I > > > > > started the node again, I saw these unexpected messages: > > > > > > > > > > Feb 09 14:50:18 h18 pacemaker‑fenced[31326]: warning: > > > > > received pending > > > > > action we are supposed to be the owner but it's not in our > > > > > records ‑> > > fail > > > > it > > > > > > Looks like some part of your cluster still had kept the pending > > > fence > > action > > > around when h18 was fencing h16. Can be that the node wasn't > > > around > > > when this was successful or it can have to do with an issue we > > > had > > recently > > > > The node definitely was "around" when h16 had been fenced, so it > > must be the > > other rerason (lingering around). > > > > > that in certain cases pending fencing actions weren't properly > > > deleted. > > > This part of the code got a major overhaul recently and the code- > > > parts > > > referred to by e.g. the assertion aren't there anymore. > > > That we are seeing this assertion makes me think, you hit the > > > case > > > with the lingering pending fencing actions (think the lingering > > > one is a > > > relayed one and looks a bit different than a plain one and thus > > > might > > > trigger > > > the assertion). > > > > > > Klaus > > > > > Feb 09 14:50:18 h18 pacemaker‑fenced[31326]: error: > > > > > Operation 'reboot' > > > > > targeting h16 on <no‑one> for pacemaker‑ > > > > > [email protected]: No > > route > > > > to > > > > > host > > > > > Feb 09 14:50:18 h18 pacemaker‑fenced[31326]: error: > > > > > > > > stonith_construct_reply: > > > > > Triggered assert at fenced_commands.c:2363 : request != NULL > > > > > Feb 09 14:50:18 h18 pacemaker‑fenced[31326]: warning: Can't > > > > > create a > > sane > > > > > reply > > > > > Feb 09 14:50:18 h18 pacemaker‑controld[31330]: notice: Peer > > > > > h16 was not > > > > > terminated (reboot) by <anyone> on behalf of > > > > > pacemaker‑controld.9087: > > No > > > > route > > > > > to host > > > > > > > > > > On the "No route to host": I could ping h16 from h18 using > > > > > the host name > > > > > without any problem. > > > > > > > > > > Two points: > > > > > Why would h18 think h16 should be fenced? > > > > > The gailed asserztion looks like a programming error. > > > > > > > > "failed assertion", sorry! > > > > > > > > > Explanations? > > > > > > > > > > Regards, > > > > > Ulrich > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Manage your subscription: > > > > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > > > > > > > ClusterLabs home: https://www.clusterlabs.org/ > > > > > > > > > > > > _______________________________________________ > > > > Manage your subscription: > > > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > > > > > ClusterLabs home: https://www.clusterlabs.org/ > > > > > > _______________________________________________ > > > Manage your subscription: > > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > > > ClusterLabs home: https://www.clusterlabs.org/ > > > > > > > > > > > > _______________________________________________ > Manage your subscription: > https://lists.clusterlabs.org/mailman/listinfo/users > > ClusterLabs home: https://www.clusterlabs.org/ -- Ken Gaillot <[email protected]> _______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
