Looks like the only way around this would be to add more nodes and set minimum masters to ensure a majority quorum.
Thanks. Gourav On Monday, 4 May 2015 12:02:27 UTC+5:30, Jason Wee wrote: > > why must you have only two nodes, would it be possible to add one more > nodes so split brain will not become an issue? > > jason > > On Mon, May 4, 2015 at 2:20 PM, Mark Walkom <markw...@gmail.com > <javascript:>> wrote: > >> Your nodes aren't in different DCs are they? If so this is why we don't >> support such setups, because ES is latency sensitive and these sorts of >> things can happen very easily when your network is unreliable. >> >> They don't try to ping other nodes because you only have two, and if they >> lose contact with one another then they both assume they are masters and >> create their own cluster. Masters don't ping other nodes at random and see >> if they should be joining a different cluster. >> >> Logically there is no difference between a primary and a replica shard, >> the only physical difference is a flag that tells the cluster state which >> is which. This is why ES will never assign a primary and it's applicable >> replica to the same node. >> >> >> You cannot get around the root of your problem unless you add another >> node to and set min masters to ensure a majority quorum. >> >> On 4 May 2015 at 15:27, Gourav H Dhelaria <gouravd...@gmail.com >> <javascript:>> wrote: >> >>> >>> 1) After network goes down, they loose communication with each other. >>> After that, they are becoming split. >>> 2) They both think they are masters. Even if they think they are >>> masters, shouldn't the ping happen to see if there are other nodes in the >>> cluster ? >>> 3) Number of replicas is set to 1. If ES doesn't differentiate, why are >>> some shards primary and others replica ? >>> >>> >>> On Monday, 4 May 2015 10:48:24 UTC+5:30, Mark Walkom wrote: >>>> >>>> >>>> 1. Why are they becoming split anyway? GC, other load, network? >>>> 2. Not if they both think they are masters. >>>> 3. Are you running replicas? If so ES doesn't really differentiate >>>> between the two. >>>> >>>> >>>> On 4 May 2015 at 15:03, Gourav H Dhelaria <gouravd...@gmail.com> wrote: >>>> >>>>> Version: 1.4. >>>>> Say there are 2 nodes X and Y, both capable of becoming master. >>>>> When network goes down, both nodes get disconnected from each other >>>>> and assume the responsibility of master. >>>>> When network is restored, they don't ping each other and form a >>>>> cluster. >>>>> >>>>> Elasticsearch service has to be restarted on any one of the nodes for >>>>> them to form a cluster. Even after they form a cluster, all primary >>>>> shards >>>>> remain on one node ( on which the service was restarted ), and all >>>>> replica >>>>> shards are on the other node. >>>>> >>>>> >>>>> This document >>>>> >>>>> >>>>> http://www.elastic.co/guide/en/elasticsearch/guide/current/_important_configuration_changes.html >>>>> >>>>> >>>>> mentions that there has to be an uneven number of master eligible >>>>> nodes. >>>>> >>>>> >>>>> Queries: >>>>> >>>>> >>>>> 1) Is there a way of avoiding split brain problem in 2 node cluster ? >>>>> >>>>> 2) After network is restored, shouldn't the nodes ping each other and >>>>> form a cluster ? >>>>> >>>>> 3) After the service is restarted to form the cluster, why don't the >>>>> primary shards get distributed on both the nodes ? >>>>> >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Gourav >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elasticsearch" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to elasticsearc...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elasticsearch/d15234b3-0ea1-4390-b136-2f02f69cd3f5%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/elasticsearch/d15234b3-0ea1-4390-b136-2f02f69cd3f5%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to elasticsearc...@googlegroups.com <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/bf7a9953-bc87-4b96-843d-7bff5899855f%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/bf7a9953-bc87-4b96-843d-7bff5899855f%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to elasticsearc...@googlegroups.com <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8nc1A_vmQ_vyG4fq2uNqFA9kZO%2BT_Y4ed0e6wwPM7ztA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8nc1A_vmQ_vyG4fq2uNqFA9kZO%2BT_Y4ed0e6wwPM7ztA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ba2982b3-fe59-4d09-8739-43c0384cf901%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
