Re: [ClusterLabs] Upgrade corosync problem

On 06/07/18 15:25 +0200, Salvatore D'angelo wrote: > On 6 Jul 2018, at 14:40, Christine Caulfield wrote: >> Yes. you can't randomly swap in and out hand-compiled libqb versions. >> Find one that works and stick to it. It's an annoying 'feature' of newer >> linkers that we had to workaround in

Re: [ClusterLabs] Upgrade corosync problem

Hi, Thanks to reply. The problem is opposite to what you are saying. When I build corosync with old libqb and I verified the new updated node worked properly I updated with new libqb hand-compiled and it works fine. But in a normale upgrade procedure I first build libqb (removing first the old

Re: [ClusterLabs] Upgrade corosync problem

On 06/07/18 13:24, Salvatore D'angelo wrote: > Hi All, > > The option --ulimit memlock=536870912 worked fine. > > I have now another strange issue. The upgrade without updating libqb > (leaving the 0.16.0) worked fine. > If after the upgrade I stop pacemaker and corosync, I download the > latest

Re: [ClusterLabs] Upgrade corosync problem

Here some strike of corosync: execve("/usr/sbin/corosync", ["corosync"], [/* 21 vars */]) = 0 brk(0) = 0x563b1f774000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file

Re: [ClusterLabs] Upgrade corosync problem

Hi All, The option --ulimit memlock=536870912 worked fine. I have now another strange issue. The upgrade without updating libqb (leaving the 0.16.0) worked fine. If after the upgrade I stop pacemaker and corosync, I download the latest libqb version:

Re: [ClusterLabs] Upgrade corosync problem

On 03/07/18 07:53, Jan Pokorný wrote: > On 02/07/18 17:19 +0200, Salvatore D'angelo wrote: >> Today I tested the two suggestions you gave me. Here what I did. >> In the script where I create my 5 machines cluster (I use three >> nodes for pacemaker PostgreSQL cluster and two nodes for glusterfs

Re: [ClusterLabs] Upgrade corosync problem

On 02/07/18 17:19 +0200, Salvatore D'angelo wrote: > Today I tested the two suggestions you gave me. Here what I did. > In the script where I create my 5 machines cluster (I use three > nodes for pacemaker PostgreSQL cluster and two nodes for glusterfs > that we use for database backup and WAL

Re: [ClusterLabs] Upgrade corosync problem

Hi All, Today I tested the two suggestions you gave me. Here what I did. In the script where I create my 5 machines cluster (I use three nodes for pacemaker PostgreSQL cluster and two nodes for glusterfs that we use for database backup and WAL files). FIRST TEST —— I added the

Re: [ClusterLabs] Upgrade corosync problem

On 29/06/18 17:20, Jan Pokorný wrote: > On 29/06/18 10:00 +0100, Christine Caulfield wrote: >> On 27/06/18 08:35, Salvatore D'angelo wrote: >>> One thing that I do not understand is that I tried to compare corosync >>> 2.3.5 (the old version that worked fine) and 2.4.4 to understand >>>

Re: [ClusterLabs] Upgrade corosync problem

Hi everyone, Thanks for suggestion. Yesterday in Rome was City Holiday and with week end I think I’ll try all your proposal Monday morning when I go back to office. Thanks again for support I appreciate it a lot. > On 29 Jun 2018, at 18:20, Jan Pokorný wrote: > > On 29/06/18 10:00 +0100,

Re: [ClusterLabs] Upgrade corosync problem

On 29/06/18 19:13 +0200, Salvatore D'angelo wrote: > Good to know. I'll try it. I'll try to work on VM too. If that won't work, you can also try: docker run ... --ulimit memlock=33554432 ... where 32768 (kiB) may still be not enough (assuming the default of 16384), hard to say, since proper

Re: [ClusterLabs] Upgrade corosync problem

Good to know. I'll try it. I'll try to work on VM too. Il Ven 29 Giu 2018, 5:46 PM Jan Pokorný ha scritto: > On 26/06/18 11:03 +0200, Salvatore D'angelo wrote: > > Yes, sorry you’re right I could find it by myself. > > However, I did the following: > > > > 1. Added the line you suggested to

Re: [ClusterLabs] Upgrade corosync problem

On 29/06/18 10:00 +0100, Christine Caulfield wrote: > On 27/06/18 08:35, Salvatore D'angelo wrote: >> One thing that I do not understand is that I tried to compare corosync >> 2.3.5 (the old version that worked fine) and 2.4.4 to understand >> differences but I haven’t found anything related to

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 11:03 +0200, Salvatore D'angelo wrote: > Yes, sorry you’re right I could find it by myself. > However, I did the following: > > 1. Added the line you suggested to /etc/fstab > 2. mount -o remount /dev/shm > 3. Now I correctly see /dev/shm of 512M with df -h > Filesystem Size

Re: [ClusterLabs] Upgrade corosync problem

On 27/06/18 08:35, Salvatore D'angelo wrote: > Hi, > > Thanks for reply and detailed explaination. I am not using the > —network=host option. > I have a docker image based on Ubuntu 14.04 where I only deploy this > additional software: > > *RUN apt-get update && apt-get install -y wget git

Re: [ClusterLabs] Upgrade corosync problem

Hi, Thanks for reply and detailed explaination. I am not using the —network=host option. I have a docker image based on Ubuntu 14.04 where I only deploy this additional software: RUN apt-get update && apt-get install -y wget git xz-utils openssh-server \

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 17:56 +0200, Salvatore D'angelo wrote: > I did another test. I modified docker container in order to be able to run > strace. > Running strace corosync-quorumtool -ps I got the following: > [snipped] > connect(5, {sa_family=AF_LOCAL, sun_path=@"cfg"}, 110) = 0 > setsockopt(5,

Re: [ClusterLabs] Upgrade corosync problem

I noticed that corosync 2.4.4 depends on the following libraries: https://launchpad.net/ubuntu/+source/corosync/2.4.4-3 I imagine that all the corosync-* and libcorosync-* libraries are build from the corosync build, so I should have them.

Re: [ClusterLabs] Upgrade corosync problem

corosync 2.3.5 and libqb 0.16.0 > On 26 Jun 2018, at 14:08, Christine Caulfield wrote: > > On 26/06/18 12:16, Salvatore D'angelo wrote: >> libqb update to 1.0.3 but same issue. >> >> I know corosync has also these dependencies nspr and nss3. I updated >> them using apt-get install, here the

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 12:16, Salvatore D'angelo wrote: > libqb update to 1.0.3 but same issue. > > I know corosync has also these dependencies nspr and nss3. I updated > them using apt-get install, here the version installed: > >    libnspr4, libnspr4-dev  2:4.13.1-0ubuntu0.14.04.1 >    libnss3, 

Re: [ClusterLabs] Upgrade corosync problem

libqb update to 1.0.3 but same issue. I know corosync has also these dependencies nspr and nss3. I updated them using apt-get install, here the version installed: libnspr4, libnspr4-dev 2:4.13.1-0ubuntu0.14.04.1 libnss3, libnss3-dev, libnss3-nssb 2:3.28.4-0ubuntu0.14.04.3

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 11:24, Salvatore D'angelo wrote: > Hi, > > I have tried with: > 0.16.0.real-1ubuntu4 > 0.16.0.real-1ubuntu5 > > which version should I try? Hmm both of those are actually quite old! maybe a newer one? Chrissie > >> On 26 Jun 2018, at 12:03, Christine Caulfield >

Re: [ClusterLabs] Upgrade corosync problem

Hi, I have tried with: 0.16.0.real-1ubuntu4 0.16.0.real-1ubuntu5 which version should I try? > On 26 Jun 2018, at 12:03, Christine Caulfield wrote: > > On 26/06/18 11:00, Salvatore D'angelo wrote: >> Consider that the container is the same when corosync 2.3.5 run. >> If it is something

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 11:00, Salvatore D'angelo wrote: > Consider that the container is the same when corosync 2.3.5 run. > If it is something related to the container probably the 2.4.4 > introduced a feature that has an impact on container. > Should be something related to libqb according to the code. >

Re: [ClusterLabs] Upgrade corosync problem

Consider that the container is the same when corosync 2.3.5 run. If it is something related to the container probably the 2.4.4 introduced a feature that has an impact on container. Should be something related to libqb according to the code. Anyone can help? > On 26 Jun 2018, at 11:56, Christine

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 10:35, Salvatore D'angelo wrote: > Sorry after the command: > > corosync-quorumtool -ps > > the error in log are still visible. Looking at the source code it seems > problem is at this line: > https://github.com/corosync/corosync/blob/master/tools/corosync-quorumtool.c > >     if

Re: [ClusterLabs] Upgrade corosync problem

Sorry after the command: corosync-quorumtool -ps the error in log are still visible. Looking at the source code it seems problem is at this line: https://github.com/corosync/corosync/blob/master/tools/corosync-quorumtool.c

Re: [ClusterLabs] Upgrade corosync problem

Yes, sorry you’re right I could find it by myself.However, I did the following:1. Added the line you suggested to /etc/fstab2. mount -o remount /dev/shm3. Now I correctly see /dev/shm of 512M with df -hFilesystem      Size  Used Avail Use% Mounted onoverlay          63G   11G   49G  19% /tmpfs     

Re: [ClusterLabs] Upgrade corosync problem

On 26/06/18 09:40, Salvatore D'angelo wrote: > Hi, > > Yes, > > I am reproducing only the required part for test. I think the original > system has a larger shm. The problem is that I do not know exactly how > to change it. > I tried the following steps, but I have the impression I didn’t >

Re: [ClusterLabs] Upgrade corosync problem

Hi, Yes, I am reproducing only the required part for test. I think the original system has a larger shm. The problem is that I do not know exactly how to change it. I tried the following steps, but I have the impression I didn’t performed the right one: 1. remove everything under /tmp 2.

Re: [ClusterLabs] Upgrade corosync problem

On 25/06/18 20:41, Salvatore D'angelo wrote: > Hi, > > Let me add here one important detail. I use Docker for my test with 5 > containers deployed on my Mac. > Basically the team that worked on this project installed the cluster on soft > layer bare metal. > The PostgreSQL cluster was hard to

Re: [ClusterLabs] Upgrade corosync problem

Hi, Let me add here one important detail. I use Docker for my test with 5 containers deployed on my Mac. Basically the team that worked on this project installed the cluster on soft layer bare metal. The PostgreSQL cluster was hard to test and if a misconfiguration occurred recreate the

Re: [ClusterLabs] Upgrade corosync problem

On 25/06/18 19:06 +0200, Salvatore D'angelo wrote: > Thanks for reply. I scratched my cluster and created it again and > then migrated as before. This time I uninstalled pacemaker, > corosync, crmsh and resource agents with make uninstall > > then I installed new packages. The problem is the

Re: [ClusterLabs] Upgrade corosync problem

Hi,Thanks for reply. I scratched my cluster and created it again and then migrated as before. This time I uninstalled pacemaker, corosync, crmsh and resource agents withmake uninstallthen I installed new packages. The problem is the same, when I launch:corosync-quorumtool -psI got: Cannot

Re: [ClusterLabs] Upgrade corosync problem

On 22/06/18 11:23, Salvatore D'angelo wrote: > Hi, > Here the log: > > > [17323] pg1 corosyncerror [QB] couldn't create circular mmap on /dev/shm/qb-cfg-event-17324-17334-23-data [17323] pg1 corosyncerror [QB] qb_rb_open:cfg-event-17324-17334-23: Resource temporarily unavailable

Re: [ClusterLabs] Upgrade corosync problem

Hi, Here the log: corosync.log Description: Binary data > On 22 Jun 2018, at 12:10, Christine Caulfield wrote: > > On 22/06/18 10:39, Salvatore D'angelo wrote: >> Hi, >> >> Can you tell me exactly which log you need. I’ll provide you as soon as >> possible. >> >> Regarding some settings,

Re: [ClusterLabs] Upgrade corosync problem

On 22/06/18 10:39, Salvatore D'angelo wrote: > Hi, > > Can you tell me exactly which log you need. I’ll provide you as soon as > possible. > > Regarding some settings, I am not the original author of this cluster. People > created it left the company I am working with and I inerithed the code

Re: [ClusterLabs] Upgrade corosync problem

Hi, Can you tell me exactly which log you need. I’ll provide you as soon as possible. Regarding some settings, I am not the original author of this cluster. People created it left the company I am working with and I inerithed the code and sometime I do not know why some settings are used. The

Re: [ClusterLabs] Upgrade corosync problem

On 22/06/18 10:14, Salvatore D'angelo wrote: > Hi Christine, > > Thanks for reply. Let me add few details. When I run the corosync > service I se the corosync process running. If I stop it and run: > > corosync -f  > > I see three warnings: > warning [MAIN  ] interface section bindnetaddr is

Re: [ClusterLabs] Upgrade corosync problem

Hi Christine, Thanks for reply. Let me add few details. When I run the corosync service I se the corosync process running. If I stop it and run: corosync -f I see three warnings: warning [MAIN ] interface section bindnetaddr is used together with nodelist. Nodelist one is going to be used.

Re: [ClusterLabs] Upgrade corosync problem

On 21/06/18 16:16, Salvatore D'angelo wrote: > Hi, > > I upgraded my PostgreSQL/Pacemaker cluster with these versions. > Pacemaker 1.1.14 -> 1.1.18 > Corosync 2.3.5 -> 2.4.4 > Crmsh 2.2.0 -> 3.0.1 > Resource agents 3.9.7 -> 4.1.1 > > I started on a first node  (I am trying one node at a time

[ClusterLabs] Upgrade corosync problem

Hi, I upgraded my PostgreSQL/Pacemaker cluster with these versions. Pacemaker 1.1.14 -> 1.1.18 Corosync 2.3.5 -> 2.4.4 Crmsh 2.2.0 -> 3.0.1 Resource agents 3.9.7 -> 4.1.1 I started on a first node (I am trying one node at a time upgrade). On a PostgreSQL slave node I did: crm node standby