Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
man votequorum auto_tie_breaker: 1 allows you to have quorum with 50%, yet if for example Aside (node with lowest id) dies, B side is 50% but won't be able to bring back the resources as the node with lowest id is in A side.If you want to avoid that, you can bring a qdevice on a VM in third location (even in a cloud nearby). Best Regards,Strahil Nikolov On Fri, Feb 25, 2022 at 20:10, Viet Nguyen wrote: ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Hi, Thank you so much for the answer. It seems to me that the one option I am having is one big cluster with 4 nodes. However, i still can not understand how i could solve the issue when one site with 2 nodes is down, then the other site along does not have quorum so it does not work... Can you please explain more on the approach for one big cluster? I am opened to another other solutions either commercial or open-source if available. Regards, Viet On Thu, 24 Feb 2022 at 18:22, Jan Friesse wrote: > Hi, > > On 24/02/2022 14:19, Viet Nguyen wrote: > > Hi, > > > > Thank you so much! Would you please advise more on this following case: > > > > The cluster I am trying to setup is Postgresql with replication streaming > > with PAF. So, it will decide one node as a master and 3 standby nodes. > > > > So, with this, from what I understand from Postgresql, having 2 > independent > > clusters (one in site A, one in site B) is not possible. I have to go > with > > one single cluster with 4 notes located in 2 different locations (site A > > and site B). > > > > Then, my question is: > > > > 1. Does the booth ticket work in this setup? > > no, not really. booth basically creates cluster on top of 2+ clusters > and arbitrator. > > > 2. Is Qnetd a better option than booth ticket? > > It's neither better nor worse. Qdevice (qnetd) adds a vote(s) to the > quorum (corosync level). Booth is able to fulfill pacemaker constrain > for ticket given only to one site in automated way. > > > > 3. Is there any better way to manage this? > > If you can really use only one big cluster then probably none of booth > or qdevice is needed. > > > 4. Since we have a distributed site and arbitrator, does fencing > make it > > even more complicated? How I could solve this problem? > > fencing is "must", it doesn't make it more complicated. Probably sbd but > I have virtually no knowledge about that. > > > > > > Sorry if my questions sound silly as I am very new to this and thank > > you so much for your help. > > yw > > Regards, >Honza > > > > > Regards, > > Viet > > > > On Thu, 24 Feb 2022 at 12:17, Jan Friesse wrote: > > > >> On 24/02/2022 10:28, Viet Nguyen wrote: > >>> Hi, > >>> > >>> Thank you so so much for your help. May i ask a following up question: > >>> > >>> For the option of having one big cluster with 4 nodes without booth, > >> then, > >>> if one site (having 2 nodes) is down, then the other site does not work > >> as > >>> it does not have quorum, am I right? Even if we have a quorum voter in > >> > >> Yup, you are right > >> > >>> either site A or B, then, if the site with quorum down, then, the other > >>> site does not work. So, how can we avoid this situation as I want > >>> that if one site is down, the other site still services? > >> > >> probably only with qnetd - so basically yet again site C. > >> > >> Regards, > >> Honza > >> > >>> > >>> Regards, > >>> Viet > >>> > >>> On Wed, 23 Feb 2022 at 17:08, Jan Friesse wrote: > >>> > Viet, > > On 22/02/2022 22:37, Viet Nguyen wrote: > > Hi, > > > > Could you please help me out with this question? > > > > I have 4 nodes cluster running in the same network but in 2 different > sites > > (building A - 2 nodes and building B - 2 nodes). My objective is to > > setup HA for this cluster with pacemaker. The expectation is if a > site > >> is > > down, the other site still services. > > > >From what I could understand so far, in order to make it work, it > >> needs > to > > have booth ticket manager installed in a different location, let's > say > > building C which connects to both sites A and B. > > > > With this assumption, i would like to ask few questions: > > > > 1. Am i right that I need to setup the booth ticket manager as > a > quorum > > voter as well? > > Yes, booth (arbitrator) has to be installed on "site" C if you want to > use booth. Just keep in mind booth has nothing to do with quorum. > > > 2. What happens if the connection between site A and B is > down, > >> but > the > > connection between A and C, B and C still up? In this case, > both > site A and > > B still have the quorum as it can connect to C, but not between > >> each > other? > > If you use booth then it's not required site A to see site B. It's > then > "site" C problem to decide which site gets ticket. > > > > 3. Or is there any better way to manage 2 sites cluster, each > has > >> 2 > > nodes? And if one site is down like environmental disaster, > then, > the other > > site still services. > > Basically there are (at least) two possible solutions: > - Have one big cluster without booth and use pcmk constraints > - Have two 2 node clusters and use booth. Then each of the two node > clusters is "indepe
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Hi, Thank you so much! Would you please advise more on this following case: The cluster I am trying to setup is Postgresql with replication streaming with PAF. So, it will decide one node as a master and 3 standby nodes. So, with this, from what I understand from Postgresql, having 2 independent clusters (one in site A, one in site B) is not possible. I have to go with one single cluster with 4 notes located in 2 different locations (site A and site B). Then, my question is: 1. Does the booth ticket work in this setup? 2. Is Qnetd a better option than booth ticket? 3. Is there any better way to manage this? 4. Since we have a distributed site and arbitrator, does fencing make it even more complicated? How I could solve this problem? Sorry if my questions sound silly as I am very new to this and thank you so much for your help. Regards, Viet On Thu, 24 Feb 2022 at 12:17, Jan Friesse wrote: > On 24/02/2022 10:28, Viet Nguyen wrote: > > Hi, > > > > Thank you so so much for your help. May i ask a following up question: > > > > For the option of having one big cluster with 4 nodes without booth, > then, > > if one site (having 2 nodes) is down, then the other site does not work > as > > it does not have quorum, am I right? Even if we have a quorum voter in > > Yup, you are right > > > either site A or B, then, if the site with quorum down, then, the other > > site does not work. So, how can we avoid this situation as I want > > that if one site is down, the other site still services? > > probably only with qnetd - so basically yet again site C. > > Regards, >Honza > > > > > Regards, > > Viet > > > > On Wed, 23 Feb 2022 at 17:08, Jan Friesse wrote: > > > >> Viet, > >> > >> On 22/02/2022 22:37, Viet Nguyen wrote: > >>> Hi, > >>> > >>> Could you please help me out with this question? > >>> > >>> I have 4 nodes cluster running in the same network but in 2 different > >> sites > >>> (building A - 2 nodes and building B - 2 nodes). My objective is to > >>> setup HA for this cluster with pacemaker. The expectation is if a site > is > >>> down, the other site still services. > >>> > >>> From what I could understand so far, in order to make it work, it > needs > >> to > >>> have booth ticket manager installed in a different location, let's say > >>> building C which connects to both sites A and B. > >>> > >>> With this assumption, i would like to ask few questions: > >>> > >>> 1. Am i right that I need to setup the booth ticket manager as a > >> quorum > >>> voter as well? > >> > >> Yes, booth (arbitrator) has to be installed on "site" C if you want to > >> use booth. Just keep in mind booth has nothing to do with quorum. > >> > >>> 2. What happens if the connection between site A and B is down, > but > >> the > >>> connection between A and C, B and C still up? In this case, both > >> site A and > >>> B still have the quorum as it can connect to C, but not between > each > >> other? > >> > >> If you use booth then it's not required site A to see site B. It's then > >> "site" C problem to decide which site gets ticket. > >> > >> > >>> 3. Or is there any better way to manage 2 sites cluster, each has > 2 > >>> nodes? And if one site is down like environmental disaster, then, > >> the other > >>> site still services. > >> > >> Basically there are (at least) two possible solutions: > >> - Have one big cluster without booth and use pcmk constraints > >> - Have two 2 node clusters and use booth. Then each of the two node > >> clusters is "independent" (have its own quorum) and each of the cluster > >> runs booth (site) as a cluster resource + "site" C running booth > >> (arbitrator) > >> > >> Regards, > >> Honza > >> > >>> > >>> > >>> Thank you so much for your help! > >>> Regards, > >>> Viet > >>> > >>> > >>> ___ > >>> Manage your subscription: > >>> https://lists.clusterlabs.org/mailman/listinfo/users > >>> > >>> ClusterLabs home: https://www.clusterlabs.org/ > >>> > >> > >> > > > > ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Hi, Thank you so so much for your help. May i ask a following up question: For the option of having one big cluster with 4 nodes without booth, then, if one site (having 2 nodes) is down, then the other site does not work as it does not have quorum, am I right? Even if we have a quorum voter in either site A or B, then, if the site with quorum down, then, the other site does not work. So, how can we avoid this situation as I want that if one site is down, the other site still services? Regards, Viet On Wed, 23 Feb 2022 at 17:08, Jan Friesse wrote: > Viet, > > On 22/02/2022 22:37, Viet Nguyen wrote: > > Hi, > > > > Could you please help me out with this question? > > > > I have 4 nodes cluster running in the same network but in 2 different > sites > > (building A - 2 nodes and building B - 2 nodes). My objective is to > > setup HA for this cluster with pacemaker. The expectation is if a site is > > down, the other site still services. > > > > From what I could understand so far, in order to make it work, it needs > to > > have booth ticket manager installed in a different location, let's say > > building C which connects to both sites A and B. > > > > With this assumption, i would like to ask few questions: > > > > 1. Am i right that I need to setup the booth ticket manager as a > quorum > > voter as well? > > Yes, booth (arbitrator) has to be installed on "site" C if you want to > use booth. Just keep in mind booth has nothing to do with quorum. > > > 2. What happens if the connection between site A and B is down, but > the > > connection between A and C, B and C still up? In this case, both > site A and > > B still have the quorum as it can connect to C, but not between each > other? > > If you use booth then it's not required site A to see site B. It's then > "site" C problem to decide which site gets ticket. > > > > 3. Or is there any better way to manage 2 sites cluster, each has 2 > > nodes? And if one site is down like environmental disaster, then, > the other > > site still services. > > Basically there are (at least) two possible solutions: > - Have one big cluster without booth and use pcmk constraints > - Have two 2 node clusters and use booth. Then each of the two node > clusters is "independent" (have its own quorum) and each of the cluster > runs booth (site) as a cluster resource + "site" C running booth > (arbitrator) > > Regards, >Honza > > > > > > > Thank you so much for your help! > > Regards, > > Viet > > > > > > ___ > > Manage your subscription: > > https://lists.clusterlabs.org/mailman/listinfo/users > > > > ClusterLabs home: https://www.clusterlabs.org/ > > > > ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Hi, On 24/02/2022 14:19, Viet Nguyen wrote: Hi, Thank you so much! Would you please advise more on this following case: The cluster I am trying to setup is Postgresql with replication streaming with PAF. So, it will decide one node as a master and 3 standby nodes. So, with this, from what I understand from Postgresql, having 2 independent clusters (one in site A, one in site B) is not possible. I have to go with one single cluster with 4 notes located in 2 different locations (site A and site B). Then, my question is: 1. Does the booth ticket work in this setup? no, not really. booth basically creates cluster on top of 2+ clusters and arbitrator. 2. Is Qnetd a better option than booth ticket? It's neither better nor worse. Qdevice (qnetd) adds a vote(s) to the quorum (corosync level). Booth is able to fulfill pacemaker constrain for ticket given only to one site in automated way. 3. Is there any better way to manage this? If you can really use only one big cluster then probably none of booth or qdevice is needed. 4. Since we have a distributed site and arbitrator, does fencing make it even more complicated? How I could solve this problem? fencing is "must", it doesn't make it more complicated. Probably sbd but I have virtually no knowledge about that. Sorry if my questions sound silly as I am very new to this and thank you so much for your help. yw Regards, Honza Regards, Viet On Thu, 24 Feb 2022 at 12:17, Jan Friesse wrote: On 24/02/2022 10:28, Viet Nguyen wrote: Hi, Thank you so so much for your help. May i ask a following up question: For the option of having one big cluster with 4 nodes without booth, then, if one site (having 2 nodes) is down, then the other site does not work as it does not have quorum, am I right? Even if we have a quorum voter in Yup, you are right either site A or B, then, if the site with quorum down, then, the other site does not work. So, how can we avoid this situation as I want that if one site is down, the other site still services? probably only with qnetd - so basically yet again site C. Regards, Honza Regards, Viet On Wed, 23 Feb 2022 at 17:08, Jan Friesse wrote: Viet, On 22/02/2022 22:37, Viet Nguyen wrote: Hi, Could you please help me out with this question? I have 4 nodes cluster running in the same network but in 2 different sites (building A - 2 nodes and building B - 2 nodes). My objective is to setup HA for this cluster with pacemaker. The expectation is if a site is down, the other site still services. From what I could understand so far, in order to make it work, it needs to have booth ticket manager installed in a different location, let's say building C which connects to both sites A and B. With this assumption, i would like to ask few questions: 1. Am i right that I need to setup the booth ticket manager as a quorum voter as well? Yes, booth (arbitrator) has to be installed on "site" C if you want to use booth. Just keep in mind booth has nothing to do with quorum. 2. What happens if the connection between site A and B is down, but the connection between A and C, B and C still up? In this case, both site A and B still have the quorum as it can connect to C, but not between each other? If you use booth then it's not required site A to see site B. It's then "site" C problem to decide which site gets ticket. 3. Or is there any better way to manage 2 sites cluster, each has 2 nodes? And if one site is down like environmental disaster, then, the other site still services. Basically there are (at least) two possible solutions: - Have one big cluster without booth and use pcmk constraints - Have two 2 node clusters and use booth. Then each of the two node clusters is "independent" (have its own quorum) and each of the cluster runs booth (site) as a cluster resource + "site" C running booth (arbitrator) Regards, Honza Thank you so much for your help! Regards, Viet ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
On Thu, Feb 24, 2022 at 1:17 PM Jan Friesse wrote: > > On 24/02/2022 10:28, Viet Nguyen wrote: > > Hi, > > > > Thank you so so much for your help. May i ask a following up question: > > > > For the option of having one big cluster with 4 nodes without booth, then, > > if one site (having 2 nodes) is down, then the other site does not work as > > it does not have quorum, am I right? Even if we have a quorum voter in > > Yup, you are right > > > either site A or B, then, if the site with quorum down, then, the other > > site does not work. So, how can we avoid this situation as I want > > that if one site is down, the other site still services? > > probably only with qnetd - so basically yet again site C. > The problem with multisite cluster is not quorum (which is not actually needed to run pacemaker) but fencing. One site cannot take over resources until another site is fenced and if another site is completely down fencing does not work. So qnetd does not really help here (except with suicide self fencing). If network conditions allow, it is better to have iSCSI target on the third site and use SBD disk heartbeat. Self fencing requires SBD anyway. ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
On 24/02/2022 10:28, Viet Nguyen wrote: Hi, Thank you so so much for your help. May i ask a following up question: For the option of having one big cluster with 4 nodes without booth, then, if one site (having 2 nodes) is down, then the other site does not work as it does not have quorum, am I right? Even if we have a quorum voter in Yup, you are right either site A or B, then, if the site with quorum down, then, the other site does not work. So, how can we avoid this situation as I want that if one site is down, the other site still services? probably only with qnetd - so basically yet again site C. Regards, Honza Regards, Viet On Wed, 23 Feb 2022 at 17:08, Jan Friesse wrote: Viet, On 22/02/2022 22:37, Viet Nguyen wrote: Hi, Could you please help me out with this question? I have 4 nodes cluster running in the same network but in 2 different sites (building A - 2 nodes and building B - 2 nodes). My objective is to setup HA for this cluster with pacemaker. The expectation is if a site is down, the other site still services. From what I could understand so far, in order to make it work, it needs to have booth ticket manager installed in a different location, let's say building C which connects to both sites A and B. With this assumption, i would like to ask few questions: 1. Am i right that I need to setup the booth ticket manager as a quorum voter as well? Yes, booth (arbitrator) has to be installed on "site" C if you want to use booth. Just keep in mind booth has nothing to do with quorum. 2. What happens if the connection between site A and B is down, but the connection between A and C, B and C still up? In this case, both site A and B still have the quorum as it can connect to C, but not between each other? If you use booth then it's not required site A to see site B. It's then "site" C problem to decide which site gets ticket. 3. Or is there any better way to manage 2 sites cluster, each has 2 nodes? And if one site is down like environmental disaster, then, the other site still services. Basically there are (at least) two possible solutions: - Have one big cluster without booth and use pcmk constraints - Have two 2 node clusters and use booth. Then each of the two node clusters is "independent" (have its own quorum) and each of the cluster runs booth (site) as a cluster resource + "site" C running booth (arbitrator) Regards, Honza Thank you so much for your help! Regards, Viet ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
Re: [ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Viet, On 22/02/2022 22:37, Viet Nguyen wrote: Hi, Could you please help me out with this question? I have 4 nodes cluster running in the same network but in 2 different sites (building A - 2 nodes and building B - 2 nodes). My objective is to setup HA for this cluster with pacemaker. The expectation is if a site is down, the other site still services. From what I could understand so far, in order to make it work, it needs to have booth ticket manager installed in a different location, let's say building C which connects to both sites A and B. With this assumption, i would like to ask few questions: 1. Am i right that I need to setup the booth ticket manager as a quorum voter as well? Yes, booth (arbitrator) has to be installed on "site" C if you want to use booth. Just keep in mind booth has nothing to do with quorum. 2. What happens if the connection between site A and B is down, but the connection between A and C, B and C still up? In this case, both site A and B still have the quorum as it can connect to C, but not between each other? If you use booth then it's not required site A to see site B. It's then "site" C problem to decide which site gets ticket. 3. Or is there any better way to manage 2 sites cluster, each has 2 nodes? And if one site is down like environmental disaster, then, the other site still services. Basically there are (at least) two possible solutions: - Have one big cluster without booth and use pcmk constraints - Have two 2 node clusters and use booth. Then each of the two node clusters is "independent" (have its own quorum) and each of the cluster runs booth (site) as a cluster resource + "site" C running booth (arbitrator) Regards, Honza Thank you so much for your help! Regards, Viet ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/ ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
[ClusterLabs] Booth ticket multi-site and quorum /Pacemaker
Hi, Could you please help me out with this question? I have 4 nodes cluster running in the same network but in 2 different sites (building A - 2 nodes and building B - 2 nodes). My objective is to setup HA for this cluster with pacemaker. The expectation is if a site is down, the other site still services. >From what I could understand so far, in order to make it work, it needs to have booth ticket manager installed in a different location, let's say building C which connects to both sites A and B. With this assumption, i would like to ask few questions: 1. Am i right that I need to setup the booth ticket manager as a quorum voter as well? 2. What happens if the connection between site A and B is down, but the connection between A and C, B and C still up? In this case, both site A and B still have the quorum as it can connect to C, but not between each other? 3. Or is there any better way to manage 2 sites cluster, each has 2 nodes? And if one site is down like environmental disaster, then, the other site still services. Thank you so much for your help! Regards, Viet ___ Manage your subscription: https://lists.clusterlabs.org/mailman/listinfo/users ClusterLabs home: https://www.clusterlabs.org/
