Hello again Ulrich and hello all,
Here is my configuration:
I have 3 hypervisors. Each has 1 VM with the cluster services running
in it. Those are the only cluster members.
The resources are a VIP, a custom systemd service that follows the VIP,
and mariadb service resource.
Stonith is disabled.
If there is output from a specific command that you would find helpful,
please ask me.
The test is simply to watch the behavior of the cluster when the cable
is removed from the hypervisor which runs the VM with the VIP resource
started on it. When the cable is pulled, the remaining nodes behave as
expected, meaning, the resources successfully start on other nodes. The
problem I saw is the unexpected exit of the pacemaker service, which you
can see in the log file I attached.
So, my first question. Based on the log file, can anyone tell me why
the pacemaker service exiting?
Should pacemaker be exiting every time it loses quorum? I ask because
I've performed several more tests and the behavior of pacemaker service
is inconsistent. Sometimes, it doesn't exit.
I am very confused by this inconsistency. Any insight would be
appreciated.
An interesting environmental issue. When the network cable is removed
from the hypervisor, the load average, according to "top", start going
up very quickly. I mention it because it may be affecting what I see.
Thank you for your time.
Best regards,
John
On 2026-11-02 05:46, Windl, Ulrich via Users wrote:
Hi!
I just had a short look. The problem seems to be "quorum lost". Also
it's not clear which of the three VMs making the cluster were affected
by pulling the cable from the hypervisor.
Kind regards,
Ulrich Windl
-----Original Message-----
From: Users <[email protected]> On Behalf Of john tillman
Sent: Friday, October 30, 2026 1:29 PM
To: Cluster Labs - All topics related to open-source clustering
welcomed
<[email protected]>
Subject: [EXT] [EXT] Re: [ClusterLabs] pacemaker&corosync services
shutting
off after the network cable is disconnected
Good day all!
So, I trimmed the file down to the point where I removed the cable
from
the hypervisor and it is down to 111KB. Hopefully, that is not too
obnoxious, and I apologize in advance for any annoyance.
This log is an example of what I have seen several times and
represents
the "self exit" of pacemaker. Seems like error status of 100 is
repeated. Also, I noticed that schedulerd "is unresponsive to IPC" at
the top right after the resources are stopped and it never comes back.
I hope someone may provide some insight for what is happening or
suggest
alternative debugging approaches. Any help would be appreciated.
3 node cluster with a VIP and a couple of services in the cluster.
pacemaker version 2.1.10-2
Rhel 9.8
Fencing is not enabled
Best regards,
-John
On 2026-10-22 15:56, john tillman wrote:
> Greetings Chris/All,
>
> I've had some time to get back to this issue. It is still occurring.
>
> To refresh, I have pacemaker services running in a VM. When I pull out
> the network cable from the associated hypervisor the pacemakerd process
> will exit.
>
> I think it would be helpful if I can provide the pacemaker.log file
> from when this issue occurs. What would be the best way to provide
> that log file? It's ~114KB and about 753 lines long.
>
> -John
>
>
> On 2026-09-14 10:07, JT wrote:
>> Chris,
>> Thanks for trying this. My experience has been with the cluster
>> services running on VMs, when the cable to the hypervisor is pulled.
>> The hypervisor only had one VM with cluster services on it. The
>> remaining cluster nodes were on other VMs on other hypervisor, all on
>> the same subnet.
>>
>> I'm not sure if the nmcli command being run on the VM would reproduce
>> the same effect?
>>
>> -John
>>
>> On September 11, 2026 4:12:43 PM EDT, Chris Lumens
>> <[email protected]> wrote:
>>
>>>> pacemaker 2.1.9-1 On Rhel 9.6 - No problem with network
>>>> isolation
>>>> pacemaker 2.1.10-2 On Rhel 9.8 - Problem I described
>>>
>>> I've been trying to reproduce this today, but so far no luck. I had
>>> an idea which patch might be to blame, so maybe there's another
>>> patch at fault that I haven't tried yet. Alternatively, perhaps
>>> there's something about how I am taking down the network on my VM
>>> (nmcli device down) that results in different behavior than how you
>>> are losing the network.
>>>
>>> - Chris
>> _______________________________________________
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/users
>>
>> ClusterLabs home: https://www.clusterlabs.org/
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
_______________________________________________
Manage your subscription:
https://lists.clusterlabs.org/mailman/listinfo/users
ClusterLabs home: https://www.clusterlabs.org/
