Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Thanks Gary, Also I asked for the sample configuration of raft cluster when raft servers are part of OpenSAF cluster A Raft(etcd) cluster should use different interface other then what OpenSAF is using, so it will better to document this. In general it will be good to have sample configuration about how to farm a raft cluster in the README as well as PR doc. Thanks, Ravi - Original Message - From: gary@dektech.com.au To: anders.wid...@ericsson.com, ravisekhar.ko...@oracle.com, hans.nordeb...@ericsson.com Cc: opensaf-devel@lists.sourceforge.net Sent: Monday, February 12, 2018 2:48:15 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: Review Request for doc: update overview PR for split brain prevention with consensus service [#64] Hi Ravi/Anders AndersW> This is slightly out of scope since there are many RAFT implementations, but I agree it could be a good idea to provide a sample configuration for etcd along with the sample etcd plugin. I will try to provide a sample plugin for an external etcd server, and maybe a sample plugin for another RAFT based key-value store. Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Hi Anders, In case where Raft Cluster is outside the OpenSAF Cluster Then how can the quorum concept applies to OpenSAF cluster, because raft servers are outside the OpenSAF Cluster they wont be able to determine which OpenSAF partition has more no of nodes Thanks, Ravi - Original Message - From: anders.wid...@ericsson.com To: ravisekhar.ko...@oracle.com, gary@dektech.com.au, hans.nordeb...@ericsson.com Cc: opensaf-devel@lists.sourceforge.net Sent: Thursday, February 8, 2018 3:21:15 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: Review Request for doc: update overview PR for split brain prevention with consensus service [#64] See my comments inline, marked AndersW> regards, Anders Widell On 02/08/2018 10:36 AM, Ravi Sekhar Reddy Konda wrote: > Hi Gary, > > Have query regarding quorum selection when raft servers are external to the > OpenSAF Cluster > > In the document we are saying "The consensus service uses quorum to prevent > state changes in network partitions that don't include more than half of the > nodes in the cluster" > > => This is possible if the raft server is installed on the OpenSAF Cluster > Nodes, as Raft decides which partition has more no of nodes. > but in the case where raft servers run on external nodes outside of the > OpenSAF Cluster, how the quorum is decided AndersW> If the consensus service is running on external servers then you need to have an appropriate number of them (probably three or five). Quorum is determined as the majority of these external servers, and is not in any way related to majority of the OpenSAF nodes. The consensus service will prevent split-brain within the OpenSAF cluster, but in case of a network partition it will not guarantee that the active system controller will be located in the largest partition. This situation is actually similar to the situation when you use TIPC for internal OpenSAF communication. You can have a split-brain in the TIPC network (for example due to misconfiguration or a bug in TIPC), but at the same time have full connectivity on the IP network which is used by RAFT. I think there were some review comments about this for ticket [#64] and I will write a follow-up ticket where we can address the possibility of moving the active system controller to a node in the largest network partition. > > > => If the Raft Servers are external to OpenSAF Cluster, do we need to make > any configuration so that etcd client on the OpenSAF nodes > communicates with Raft Leader > Also it will be good if we give some details about how to install and > configure raft(raft servers within and external to the opensaf cluster) AndersW> This is slightly out of scope since there are many RAFT implementations, but I agree it could be a good idea to provide a sample configuration for etcd along with the sample etcd plugin. > > Thanks, > Ravi > > -Original Message- > From: Gary Lee [mailto:gary@dektech.com.au] > Sent: Friday, January 26, 2018 11:28 AM > To: Hans Nordebäck; Anders Widell > ; Ravi Sekhar Reddy Konda > > Cc: opensaf-devel@lists.sourceforge.net > Subject: Review Request for doc: update overview PR for split brain > prevention with consensus service [#64] > > Hi > > I have updated the OpenSAF Overview PR document for ticket #64. > > Please have a look. > > https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceforge.net_p_opensaf_tickets_-5Fdiscuss_thread_0d47d4b9_5489_attachment_OpenSAF-5FOverview-5FPR.odt=DwICaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=xBh_3WtlS1YjXd3Bui_nVjh5qwhU2UamdAhSfqynLU4=xCEIb5x0gLGfoZW5uOWz23MZa6HzmOa6Vhywz3WeIQs=RF6RsX3xhby4k4PnwA8WEXCWKg0JbFyGNgaiery9iDk= > > Thanks > Gary > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Hi Ravi/Anders AndersW> This is slightly out of scope since there are many RAFT implementations, but I agree it could be a good idea to provide a sample configuration for etcd along with the sample etcd plugin. I will try to provide a sample plugin for an external etcd server, and maybe a sample plugin for another RAFT based key-value store. Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
See my comments inline, marked AndersW> regards, Anders Widell On 02/08/2018 10:36 AM, Ravi Sekhar Reddy Konda wrote: Hi Gary, Have query regarding quorum selection when raft servers are external to the OpenSAF Cluster In the document we are saying "The consensus service uses quorum to prevent state changes in network partitions that don't include more than half of the nodes in the cluster" => This is possible if the raft server is installed on the OpenSAF Cluster Nodes, as Raft decides which partition has more no of nodes. but in the case where raft servers run on external nodes outside of the OpenSAF Cluster, how the quorum is decided AndersW> If the consensus service is running on external servers then you need to have an appropriate number of them (probably three or five). Quorum is determined as the majority of these external servers, and is not in any way related to majority of the OpenSAF nodes. The consensus service will prevent split-brain within the OpenSAF cluster, but in case of a network partition it will not guarantee that the active system controller will be located in the largest partition. This situation is actually similar to the situation when you use TIPC for internal OpenSAF communication. You can have a split-brain in the TIPC network (for example due to misconfiguration or a bug in TIPC), but at the same time have full connectivity on the IP network which is used by RAFT. I think there were some review comments about this for ticket [#64] and I will write a follow-up ticket where we can address the possibility of moving the active system controller to a node in the largest network partition. => If the Raft Servers are external to OpenSAF Cluster, do we need to make any configuration so that etcd client on the OpenSAF nodes communicates with Raft Leader Also it will be good if we give some details about how to install and configure raft(raft servers within and external to the opensaf cluster) AndersW> This is slightly out of scope since there are many RAFT implementations, but I agree it could be a good idea to provide a sample configuration for etcd along with the sample etcd plugin. Thanks, Ravi -Original Message- From: Gary Lee [mailto:gary@dektech.com.au] Sent: Friday, January 26, 2018 11:28 AM To: Hans Nordebäck; Anders Widell ; Ravi Sekhar Reddy Konda Cc: opensaf-devel@lists.sourceforge.net Subject: Review Request for doc: update overview PR for split brain prevention with consensus service [#64] Hi I have updated the OpenSAF Overview PR document for ticket #64. Please have a look. https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceforge.net_p_opensaf_tickets_-5Fdiscuss_thread_0d47d4b9_5489_attachment_OpenSAF-5FOverview-5FPR.odt=DwICaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=xBh_3WtlS1YjXd3Bui_nVjh5qwhU2UamdAhSfqynLU4=xCEIb5x0gLGfoZW5uOWz23MZa6HzmOa6Vhywz3WeIQs=RF6RsX3xhby4k4PnwA8WEXCWKg0JbFyGNgaiery9iDk= Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Hi Gary, Have query regarding quorum selection when raft servers are external to the OpenSAF Cluster In the document we are saying "The consensus service uses quorum to prevent state changes in network partitions that don't include more than half of the nodes in the cluster" => This is possible if the raft server is installed on the OpenSAF Cluster Nodes, as Raft decides which partition has more no of nodes. but in the case where raft servers run on external nodes outside of the OpenSAF Cluster, how the quorum is decided => If the Raft Servers are external to OpenSAF Cluster, do we need to make any configuration so that etcd client on the OpenSAF nodes communicates with Raft Leader Also it will be good if we give some details about how to install and configure raft(raft servers within and external to the opensaf cluster) Thanks, Ravi -Original Message- From: Gary Lee [mailto:gary@dektech.com.au] Sent: Friday, January 26, 2018 11:28 AM To: Hans Nordebäck; Anders Widell ; Ravi Sekhar Reddy Konda Cc: opensaf-devel@lists.sourceforge.net Subject: Review Request for doc: update overview PR for split brain prevention with consensus service [#64] Hi I have updated the OpenSAF Overview PR document for ticket #64. Please have a look. https://urldefense.proofpoint.com/v2/url?u=https-3A__sourceforge.net_p_opensaf_tickets_-5Fdiscuss_thread_0d47d4b9_5489_attachment_OpenSAF-5FOverview-5FPR.odt=DwICaQ=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE=xBh_3WtlS1YjXd3Bui_nVjh5qwhU2UamdAhSfqynLU4=xCEIb5x0gLGfoZW5uOWz23MZa6HzmOa6Vhywz3WeIQs=RF6RsX3xhby4k4PnwA8WEXCWKg0JbFyGNgaiery9iDk= Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
Re: [devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Ack. regards, Anders Widell On 01/26/2018 06:57 AM, Gary Lee wrote: Hi I have updated the OpenSAF Overview PR document for ticket #64. Please have a look. https://sourceforge.net/p/opensaf/tickets/_discuss/thread/0d47d4b9/5489/attachment/OpenSAF_Overview_PR.odt Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel
[devel] Review Request for doc: update overview PR for split brain prevention with consensus service [#64]
Hi I have updated the OpenSAF Overview PR document for ticket #64. Please have a look. https://sourceforge.net/p/opensaf/tickets/_discuss/thread/0d47d4b9/5489/attachment/OpenSAF_Overview_PR.odt Thanks Gary -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Opensaf-devel mailing list Opensaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/opensaf-devel