Hello Reid, What solution can I use while waiting for a fix for this bug? Modify RA?
Thanks, Best regards, Thomas Cas | Technicien du support infogérance PHONE : +33 3 51 25 23 26 WEB : www.ikoula.com/en<https://www.ikoula.com/en> IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE Before printing this letter, think about the impact on the environment! [Ikoula]<https://www.ikoula.com/en> [Twitter]<https://twitter.com/ikoula_en> [Linkedin] <https://www.linkedin.com/company/ikoula> [Youtube] <http://www.youtube.fr/ikoulanet> [Pressroom] <https://pressroom.ikoula.com/> [Blog] <https://blog.ikoula.com/en> De : Thomas CAS Envoyé : vendredi 27 janvier 2023 11:28 À : Reid Wahl <nw...@redhat.com> Cc : Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org> Objet : RE: [ClusterLabs] Problem with MariaDB cluster Reid, I have activated the trace mode on the MYSQL resource on my lab and I rebooted Node2, after reboot: + echo Node1 + tr -d + master_host=Node1 + echo Node1 + [ Node1 -a Node1 != Node2 ] + ocf_log info Changing MySQL configuration to replicate from zabbixmysqlmaster01. I can see that the variable "master_host" is well filled. In production, I did the same thing and variable "master_host" is not filled: + echo + tr -d + master_host= + [ -a != Node2 ] + ocf_log info No MySQL master present - clearing replication state That is why I would like to know concretely what the variable "$OCF_RESKEY_CRM_meta_notify_master_uname" does. How is the master_host name retrieved? Best regards, Thomas Cas | Technicien du support infogérance PHONE : +33 3 51 25 23 26 WEB : www.ikoula.com/en<https://www.ikoula.com/en> IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE Before printing this letter, think about the impact on the environment! [Ikoula]<https://www.ikoula.com/en> [Twitter]<https://twitter.com/ikoula_en> [Linkedin] <https://www.linkedin.com/company/ikoula> [Youtube] <http://www.youtube.fr/ikoulanet> [Pressroom] <https://pressroom.ikoula.com/> [Blog] <https://blog.ikoula.com/en> De : Reid Wahl <nw...@redhat.com<mailto:nw...@redhat.com>> Envoyé : vendredi 27 janvier 2023 09:32 À : Thomas CAS <t...@ikoula.com<mailto:t...@ikoula.com>> Cc : Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org<mailto:users@clusterlabs.org>> Objet : Re: [ClusterLabs] Problem with MariaDB cluster On Fri, Jan 27, 2023 at 12:23 AM Thomas CAS <t...@ikoula.com<mailto:t...@ikoula.com>> wrote: Hello Reid, Thank you so much for your answer and bug report. If it is a bug, I do not understand why the problem is present in production but not on my lab which is identical? That's a good question. I'm not sure, and I haven't worked much with this resource agent. Does the lab show interesting logs at all from the resource agent during startup? I wonder if it's hitting the same "No MySQL master present" issue but *not* the error that follows. That error is coming from mysql itself, not from the resource agent. After the agent hits the "No MySQL master present" issue, it calls the unset_master() function. You can take a look at the agent script (in /usr/lib/ocf/resource.d/heartbeat/mysql by default) to see all the things that unset_master() is doing, and try to determine what's behaving differently in production vs. lab. What does the "$OCF_RESKEY_CRM_meta_notify_master_uname" variable do? (which shell command is done through this variable) Pacemaker sets it during a resource's notify operation. It's set to some information that's used only during a notify operation, and it's unset otherwise. You can see some other similar meta variables in the mysql_notify() function. The mysql_notify() function gets called during a notify operation, so those variables might be set. They won't be set when mysql_start() is called during a start operation. Best regards, Thomas Cas | Technicien du support infogérance PHONE : +33 3 51 25 23 26 WEB : www.ikoula.com/en<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570154976%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=uNarHoMr%2FuGeTV6PG6OWomI%2Fuu80OgcY2ER8IYBXv6s%3D&reserved=0> IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE Before printing this letter, think about the impact on the environment! [Ikoula]<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=h8av%2BxS5nwirxw2yo0QqWLYhrGQo4Lz%2F6ysNY6T2Ors%3D&reserved=0> [Twitter]<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fikoula_en&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=01Qxj8kQM5dPETsPR79XKdFiw6naohw8hfdxQu5aJ4w%3D&reserved=0> [Linkedin] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fikoula&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rhf%2F3YDXmJBMWRYJdg0AZ3YmkXVUVEosXpXzonOMWIE%3D&reserved=0> [Youtube] <https://fra01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.youtube.fr%2Fikoulanet&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gh3ZTTfeW0BMgSoconZDKkqjGPHvWtqyoBNBk4ovpQ%3D&reserved=0> [Pressroom] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpressroom.ikoula.com%2F&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4konuDkG%2Bu1PBTKfyeCHpz6xL%2BgkQQCoQOfbGEhdy8c%3D&reserved=0> [Blog] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NReeC%2BRXLPcjWP%2F1mf87SrQaWlZ2dVEfdCvhiKctiTo%3D&reserved=0> De : Reid Wahl <nw...@redhat.com<mailto:nw...@redhat.com>> Envoyé : jeudi 26 janvier 2023 20:31 À : Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org<mailto:users@clusterlabs.org>>; Thomas CAS <t...@ikoula.com<mailto:t...@ikoula.com>> Objet : Re: [ClusterLabs] Problem with MariaDB cluster Vous ne recevez pas souvent de courriers de la part de nw...@redhat.com<mailto:nw...@redhat.com>. Découvrez pourquoi cela est important<https://aka.ms/LearnAboutSenderIdentification> On Thu, Jan 26, 2023 at 7:39 AM Thomas CAS <t...@ikoula.com<mailto:t...@ikoula.com>> wrote: Hello, I'm having trouble with a MariaDB cluster (2 nodes, master-slave) on Debian 11. I don't know what to do anymore. Environment: Node1: OS: Debian 11 Kernel: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) Versions: resource-agents (4.7.0-1), pacemaker (2.0.5-2), corosync (3.1.2-2), mariadb (10.5.18-0+deb11u1) Node2: OS: Debian 11 Kernel: 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) Versions: resource-agents (4.7.0-1), pacemaker (2.0.5-2), corosync (3.1.2-2), mariadb (10.5.18-0+deb11u1) crm configure show as attachment. Problem: When I restart Node2 (which is a slave), it goes up correctly in the cluster: $ crm status Cluster Summary: * Stack: corosync * Current DC: Node1 (version 2.0.5-ba59be7122) - partition with quorum * Last updated: Thu Jan 26 12:04:57 2023 * Last change: Thu Jan 26 11:39:58 2023 by root via cibadmin on Node2 * 2 nodes configured * 3 resource instances configured Node List: * Online: [ Node1 Node2 ] Full List of Resources: * VIP (ocf::heartbeat:IPaddr2): Started Node1 * Clone Set: MYSQLREPLICATOR [MYSQL] (promotable): * Masters: [ Node1 ] * Slaves: [ Node2 ] But it does not retrieve the replication information. (SHOW SLAVE STATUS; returns nothing) In the Node2 logs, I can see this message that explains that replication is not taking place: Jan 25 16:29:38 mysql(MYSQL)[22862]: INFO: No MySQL master present - clearing replication state Jan 25 16:29:39 mysql(MYSQL)[22862]: WARNING: MySQL Slave IO threads currently not running. Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: MySQL Slave SQL threads currently not running. Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: See for details Jan 25 16:29:39 mysql(MYSQL)[22862]: ERROR: ERROR 1200 (HY000) at line 1: Misconfigured slave: MASTER_HOST was not set; Fix in config file or with CHANGE MASTER TO >From what I see in the following file, Node2 does not seem to find the master >name. So it clears its replication information: /usr/lib/ocf/resource.d/heartbeat/mysql master_host=`echo $OCF_RESKEY_CRM_meta_notify_master_uname|tr -d " "` if [ "$master_host" -a "$master_host" != ${NODENAME} ]; then ocf_log info "Changing MySQL configuration to replicate from $master_host." set_master start_slave if [ $? -ne 0 ]; then ocf_exit_reason "Failed to start slave" return $OCF_ERR_GENERIC fi else ocf_log info "No MySQL master present - clearing replication state" unset_master fi As it is a production environment, I performed a bare metal restore of these machines on 2 LAB machines and I have no problem... In production, there is a lot of writing but the servers are far from being saturated. Thank you in advance for all the help you can give me. Best regards, I'm sorry you've encountered this. I don't understand why the resource agent checks $OCF_RESKEY_CRM_meta_notify_master_uname during the start operation. That value gets set only during a notify operation. That looks like a bug in the resource agent. I've filed an issue against it here: https://github.com/ClusterLabs/resource-agents/issues/1839<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FClusterLabs%2Fresource-agents%2Fissues%2F1839&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1rZ5TyqG4PhgpfaJr%2FrRp9retwphLm2Ifm669sVx3p4%3D&reserved=0> Thomas Cas | Technicien du support infogérance PHONE : +33 3 51 25 23 26 WEB : www.ikoula.com/en<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=h8av%2BxS5nwirxw2yo0QqWLYhrGQo4Lz%2F6ysNY6T2Ors%3D&reserved=0> IKOULA Data Center 34 rue Pont Assy - 51100 Reims - FRANCE Before printing this letter, think about the impact on the environment! [Ikoula]<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=h8av%2BxS5nwirxw2yo0QqWLYhrGQo4Lz%2F6ysNY6T2Ors%3D&reserved=0> [Twitter]<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Fikoula_en&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=01Qxj8kQM5dPETsPR79XKdFiw6naohw8hfdxQu5aJ4w%3D&reserved=0> [Linkedin] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fikoula&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rhf%2F3YDXmJBMWRYJdg0AZ3YmkXVUVEosXpXzonOMWIE%3D&reserved=0> [Youtube] <https://fra01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.youtube.fr%2Fikoulanet&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5gh3ZTTfeW0BMgSoconZDKkqjGPHvWtqyoBNBk4ovpQ%3D&reserved=0> [Pressroom] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpressroom.ikoula.com%2F&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4konuDkG%2Bu1PBTKfyeCHpz6xL%2BgkQQCoQOfbGEhdy8c%3D&reserved=0> [Blog] <https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.ikoula.com%2Fen&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NReeC%2BRXLPcjWP%2F1mf87SrQaWlZ2dVEfdCvhiKctiTo%3D&reserved=0> _______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.clusterlabs.org%2Fmailman%2Flistinfo%2Fusers&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Q5d%2BqFrualB8vQfiCG20iIkSdVCIypAR3cUChkYx2uo%3D&reserved=0> ClusterLabs home: https://www.clusterlabs.org/<https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.clusterlabs.org%2F&data=05%7C01%7Ctcas%40ikoula.com%7C226d88b9b1754639362e08db00410952%7Ccb7a4a4ea7f747cc931f80db4a66f1c7%7C0%7C0%7C638104051570311189%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=83gTfCx%2F%2Fyor41uHtAvTeoZogsqc7WXzCDBIBU2K%2F1M%3D&reserved=0> -- Regards, Reid Wahl (He/Him) Senior Software Engineer, Red Hat RHEL High Availability - Pacemaker -- Regards, Reid Wahl (He/Him) Senior Software Engineer, Red Hat RHEL High Availability - Pacemaker
_______________________________________________ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
