Re: Cluster state yellow after upgrade to 1.0

2014-02-18 Thread Boaz Leskes
Hi Mo,

The m indicates that all nodes *may* become master. The * means that node 
is the current master. There is nothing wrong with that.

Your cluster is yellow because not all shards are started. If things don't 
sort out themselves within a couple of minutes (usually much quicker), 
something is going on. 

Cheers,
Boaz

On Monday, February 17, 2014 8:45:58 PM UTC+1, Mo wrote:

 I also see some shards in INITIALIZING
  

 [root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/shards/test2'

 test2 2 p STARTED 37382168 9.8gb 10.80.140.59

 test2 2 r INITIALIZING 10.80.140.62

 test2 0 p STARTED 37382444 9.8gb 10.80.140.59

 test2 0 r STARTED 37382444 9.8gb 10.80.140.62

 test2 3 p STARTED 37388544 9.8gb 10.80.140.59

 test2 3 r STARTED 37388544 9.8gb 10.80.140.62

 test2 1 p STARTED 37389978 9.8gb 10.80.140.59

 test2 1 r STARTED 37389978 9.8gb 10.80.140.62

 test2 4 p STARTED 37393666 9.8gb 10.80.140.59

 test2 4 r INITIALIZING 10.80.140.62


 On Mon, Feb 17, 2014 at 11:44 AM, Mohit Anchlia 
 mohita...@gmail.comjavascript:
  wrote:

 I just upgraded from .90 to 1.0 and my cluster state is showing yellow:
  
 Not sure why one node doesn't show up as m

 [root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/nodes'

 17 5 1.47 d m 10.80.140.62

 22 4 0.00 d m 10.80.140.58

 27 5 0.55 d * 10.80.140.59

 21 4 0.06 d m 10.80.140.13

  

  

 [root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/indices'

 yellow test2 5 1 186936800 0 79gb 49.4gb

 green test1 5 1 142163219 0 76.9gb 38.4gb

 [root@ip-10-80-140-59 ec2-user]#




-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/95c8d401-ddc3-4922-b2ab-47453ad879a2%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow after upgrade to 1.0

2014-02-17 Thread Mohit Anchlia
I also see some shards in INITIALIZING


[root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/shards/test2'

test2 2 p STARTED 37382168 9.8gb 10.80.140.59

test2 2 r INITIALIZING 10.80.140.62

test2 0 p STARTED 37382444 9.8gb 10.80.140.59

test2 0 r STARTED 37382444 9.8gb 10.80.140.62

test2 3 p STARTED 37388544 9.8gb 10.80.140.59

test2 3 r STARTED 37388544 9.8gb 10.80.140.62

test2 1 p STARTED 37389978 9.8gb 10.80.140.59

test2 1 r STARTED 37389978 9.8gb 10.80.140.62

test2 4 p STARTED 37393666 9.8gb 10.80.140.59

test2 4 r INITIALIZING 10.80.140.62


On Mon, Feb 17, 2014 at 11:44 AM, Mohit Anchlia mohitanch...@gmail.comwrote:

 I just upgraded from .90 to 1.0 and my cluster state is showing yellow:

 Not sure why one node doesn't show up as m

 [root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/nodes'

 17 5 1.47 d m 10.80.140.62

 22 4 0.00 d m 10.80.140.58

 27 5 0.55 d * 10.80.140.59

 21 4 0.06 d m 10.80.140.13





 [root@ip-10-80-140-59 ec2-user]# curl 'localhost:9200/_cat/indices'

 yellow test2 5 1 186936800 0 79gb 49.4gb

 green test1 5 1 142163219 0 76.9gb 38.4gb

 [root@ip-10-80-140-59 ec2-user]#


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWq6%2B6yhNHan1rMhSE2vvqTxEPbVr%2B%2BB0Y3haprAO_C%2BxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-02-10 Thread Alexander Reelsen
Hey,

not sure, if I made my point clear here:
The minimum_master_nodes is an important setting, if a master node election
is about to happen (this is not checked for each write of a document).
The write_consistency is a check, when a document is about to be stored.
Also note, that this check happens, before the data is written to the
primary shard. (This is not checked during a master election).

Hope it makes more sense now.


---Alex


On Thu, Feb 6, 2014 at 6:35 PM, Mohit Anchlia mohitanch...@gmail.comwrote:

 So the checks are made in that order? 1) enough master nodes are available
 2) Within that cluster decide how many replicas would acknowledge success
 before returning back to the client

 In other words just by having write_consistency set to quorum it doesn't
 gurantee protection against split brain.

 On Thu, Feb 6, 2014 at 1:30 AM, Alexander Reelsen a...@spinscale.dewrote:

 Hey,

 these two settings to not have a lot in common, wondering in what context
 you see them the same.

 minimum_master_nodes setting is only about the count of available nodes,
 which are eligible to become a master in order to actually have an existing
 master in a cluster.
 write_consistency qourum is actually a check when a document is indexed
 (and not about the nodes in a cluster) in order to decide if the document
 should be written at all (before it is actually indexed)

 See
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#master-election
 and
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-consistency

 Does that make sense, is this more clear now?


 --Alex



 On Mon, Feb 3, 2014 at 11:19 PM, Mohit Anchlia mohitanch...@gmail.comwrote:

 How does minimum_master_nodes differ from action.write_consistency?
 Would setting write_consistency to quorum help when minimum_master  is not
 set appropriately?

 On Thu, Jan 16, 2014 at 6:08 AM, Norberto Meijome num...@gmail.comwrote:

 Gotcha, my bad.
 On 17/01/2014 12:43 AM, joergpra...@gmail.com joergpra...@gmail.com
 wrote:

 minimum_master_nodes is a dynamic cluster setting, that means, it can
 be set via cluster update API.

 Jörg

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFgi5Wj%2Bz0P%2Bq3AJj55bTjaNhjGAzj_DHPmVfq6iRpBQ%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CACj2-4JfuGA4wAmTCjMdo2T%2B4cuY96EV1EN3Grc-pDMt%3D3xX%3DA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
  To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJHi_R_WXquKD-GmYv_fojs23h%2B-rBs-NZxNSznQwmQ%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-4K9f6dDScGMS75nZU-LYBvdQ5opi7k-nVBi6w%3DwstyA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWp6QV%2B8ySSY7PYLBiz2DEE%2B-xRvU0SnUiORJdRc1r85Fg%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-wn_YzhkiPS1F1aM_hxAioHNtmO_ndQyUOPA-_S52KFQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-02-06 Thread Alexander Reelsen
Hey,

these two settings to not have a lot in common, wondering in what context
you see them the same.

minimum_master_nodes setting is only about the count of available nodes,
which are eligible to become a master in order to actually have an existing
master in a cluster.
write_consistency qourum is actually a check when a document is indexed
(and not about the nodes in a cluster) in order to decide if the document
should be written at all (before it is actually indexed)

See
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#master-election
and
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-consistency

Does that make sense, is this more clear now?


--Alex



On Mon, Feb 3, 2014 at 11:19 PM, Mohit Anchlia mohitanch...@gmail.comwrote:

 How does minimum_master_nodes differ from action.write_consistency? Would
 setting write_consistency to quorum help when minimum_master  is not set
 appropriately?

 On Thu, Jan 16, 2014 at 6:08 AM, Norberto Meijome num...@gmail.comwrote:

 Gotcha, my bad.
 On 17/01/2014 12:43 AM, joergpra...@gmail.com joergpra...@gmail.com
 wrote:

 minimum_master_nodes is a dynamic cluster setting, that means, it can be
 set via cluster update API.

 Jörg

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFgi5Wj%2Bz0P%2Bq3AJj55bTjaNhjGAzj_DHPmVfq6iRpBQ%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CACj2-4JfuGA4wAmTCjMdo2T%2B4cuY96EV1EN3Grc-pDMt%3D3xX%3DA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJHi_R_WXquKD-GmYv_fojs23h%2B-rBs-NZxNSznQwmQ%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-4K9f6dDScGMS75nZU-LYBvdQ5opi7k-nVBi6w%3DwstyA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-02-06 Thread Mohit Anchlia
So the checks are made in that order? 1) enough master nodes are available
2) Within that cluster decide how many replicas would acknowledge success
before returning back to the client

In other words just by having write_consistency set to quorum it doesn't
gurantee protection against split brain.

On Thu, Feb 6, 2014 at 1:30 AM, Alexander Reelsen a...@spinscale.de wrote:

 Hey,

 these two settings to not have a lot in common, wondering in what context
 you see them the same.

 minimum_master_nodes setting is only about the count of available nodes,
 which are eligible to become a master in order to actually have an existing
 master in a cluster.
 write_consistency qourum is actually a check when a document is indexed
 (and not about the nodes in a cluster) in order to decide if the document
 should be written at all (before it is actually indexed)

 See
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#master-election
 and
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-consistency

 Does that make sense, is this more clear now?


 --Alex



 On Mon, Feb 3, 2014 at 11:19 PM, Mohit Anchlia mohitanch...@gmail.comwrote:

 How does minimum_master_nodes differ from action.write_consistency? Would
 setting write_consistency to quorum help when minimum_master  is not set
 appropriately?

 On Thu, Jan 16, 2014 at 6:08 AM, Norberto Meijome num...@gmail.comwrote:

 Gotcha, my bad.
 On 17/01/2014 12:43 AM, joergpra...@gmail.com joergpra...@gmail.com
 wrote:

 minimum_master_nodes is a dynamic cluster setting, that means, it can
 be set via cluster update API.

 Jörg

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFgi5Wj%2Bz0P%2Bq3AJj55bTjaNhjGAzj_DHPmVfq6iRpBQ%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CACj2-4JfuGA4wAmTCjMdo2T%2B4cuY96EV1EN3Grc-pDMt%3D3xX%3DA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrGJHi_R_WXquKD-GmYv_fojs23h%2B-rBs-NZxNSznQwmQ%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-4K9f6dDScGMS75nZU-LYBvdQ5opi7k-nVBi6w%3DwstyA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWp6QV%2B8ySSY7PYLBiz2DEE%2B-xRvU0SnUiORJdRc1r85Fg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-16 Thread joergpra...@gmail.com
minimum_master_nodes is a dynamic cluster setting, that means, it can be
set via cluster update API.

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFgi5Wj%2Bz0P%2Bq3AJj55bTjaNhjGAzj_DHPmVfq6iRpBQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-16 Thread Norberto Meijome
Gotcha, my bad.
On 17/01/2014 12:43 AM, joergpra...@gmail.com joergpra...@gmail.com
wrote:

 minimum_master_nodes is a dynamic cluster setting, that means, it can be
 set via cluster update API.

 Jörg

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHFgi5Wj%2Bz0P%2Bq3AJj55bTjaNhjGAzj_DHPmVfq6iRpBQ%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CACj2-4JfuGA4wAmTCjMdo2T%2B4cuY96EV1EN3Grc-pDMt%3D3xX%3DA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread joergpra...@gmail.com
Unicast is a nightmare for large ES deployments, with provisioning and
failures all the time. I'm used to DHCP/TFTP/PXE in my DC thanks to RedHat
so why should I waste time setting up hostnames or count hosts for ES?

Jörg


On Wed, Jan 15, 2014 at 5:17 PM, InquiringMind brian.from...@gmail.comwrote:


 One more reason to prefer unicast over multicast!



-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGdS1h-f85Tk-h8SBY92eqfXEvrGV6E0sqPNwkbXrfaxA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread Mohit Anchlia
IMHO I think ES can still be smart enough to calculate that formula
dynamically since it knows when servers are being added to the cluster,
correct? As for the node crash it's still a crash and if user wants to
decomission the node then the better way would be to explicitly run a
decomission command to indicate that nodes is not part of cluster anymore?
Is there any problem with this logic?

On Wed, Jan 15, 2014 at 9:15 AM, joergpra...@gmail.com 
joergpra...@gmail.com wrote:

 Unicast is a nightmare for large ES deployments, with provisioning and
 failures all the time. I'm used to DHCP/TFTP/PXE in my DC thanks to RedHat
 so why should I waste time setting up hostnames or count hosts for ES?

 Jörg


 On Wed, Jan 15, 2014 at 5:17 PM, InquiringMind brian.from...@gmail.comwrote:


 One more reason to prefer unicast over multicast!

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGdS1h-f85Tk-h8SBY92eqfXEvrGV6E0sqPNwkbXrfaxA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoCEg%3DL-wDNXR1mFFScDZT%2BTxUsAoyBZ31OHoLtEyo6uA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread InquiringMind
Jörg

I avoided multicast and preferred unicast based on many discussions in the 
newsgroups and other sites. In particular, the ElasticSearch Preflight 
Checklisthttp://asquera.de/opensource/2012/11/25/elasticsearch-pre-flight-checklist/.
 
Within this checklist, the sections entitled DISCOVERY and AVOIDING 
SPLIT-BRAINS were the recommendationsI that I followed. This write-up 
didn't say that mutlicast was bad, but neither have I read before that 
unicast was a nightmare. 

Whenever you get a chance, if you could describe the provisioning issues 
and failures, that would help to shed some more light on this subject.

And it does seem that somebody needs to waste time counting hosts. You 
stated (and I agree) that minimum_master_nodes must be set to N/2 + 1, and 
N must be specified by the system admin... via counting? Or is there 
something else I am not seeing?

Brian

On Wednesday, January 15, 2014 12:15:27 PM UTC-5, Jörg Prante wrote:

 Unicast is a nightmare for large ES deployments, with provisioning and 
 failures all the time. I'm used to DHCP/TFTP/PXE in my DC thanks to RedHat 
 so why should I waste time setting up hostnames or count hosts for ES?

 Jörg


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4353c8da-b73e-4175-a083-a80ea3d65f69%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread joergpra...@gmail.com
The preflight checklist is misleading in several statements:

- you can not accidentally join clusters because of multicast. It happens
when using the default cluster name

- a lot of chatter with no use is just pure ignorance of network
technology. Multicast was designed for zero config, and I love ES for zero
config. And there is no chatter if you set up a private network for ES
hosts. That is simple with a router/switch and DHCP. Whole corporation and
datacenter networks can not live without link-local config for example.

- you don't have to include the whole cluster is misleading - each data
node must always be able to connect to one of the eligible master nodes, or
the cluster may not work correctly

- the avoiding split brain section is a very bold and wrong headline. It
is promising that split-brains can be avoided by a N/2+1 setting.
Unfortunately that is nonsense, the truth is, it lowers the risk
significantly. But there can still be obscure network splits where two
halves of a split cluster may overlap, so the condition of N/2+1 can become
true for both halves.

Why do I not count hosts/nodes? There is a difference between the number of
data nodes and master eligible nodes in a cluster. I have three master
eligible nodes and set minimum_master_nodes to 3, and that's it. After that
I can start or stop additional data nodes as much as I like.

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG8Yi8KuG7%2BYa3HQEAj%2BDhPUvzoqd6aR-Gpd4iQxAF%3D%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread InquiringMind
Thank you very much, Jörg. Your explanation is clear and concise and 
greatly improves my understanding!

Brian

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/18afbcfe-fcd1-47e9-91cb-a617dc089abf%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread Mohit Anchlia
IHO, I think there is no perfect solution for any of the complex
network issues, however if we know of a config that reduces the risk
signficantly then I think we should adopt that as a default config.

On Wed, Jan 15, 2014 at 11:17 AM, joergpra...@gmail.com 
joergpra...@gmail.com wrote:

 The preflight checklist is misleading in several statements:

 - you can not accidentally join clusters because of multicast. It
 happens when using the default cluster name

 - a lot of chatter with no use is just pure ignorance of network
 technology. Multicast was designed for zero config, and I love ES for zero
 config. And there is no chatter if you set up a private network for ES
 hosts. That is simple with a router/switch and DHCP. Whole corporation and
 datacenter networks can not live without link-local config for example.

 - you don't have to include the whole cluster is misleading - each data
 node must always be able to connect to one of the eligible master nodes, or
 the cluster may not work correctly

 - the avoiding split brain section is a very bold and wrong headline. It
 is promising that split-brains can be avoided by a N/2+1 setting.
 Unfortunately that is nonsense, the truth is, it lowers the risk
 significantly. But there can still be obscure network splits where two
 halves of a split cluster may overlap, so the condition of N/2+1 can become
 true for both halves.

 Why do I not count hosts/nodes? There is a difference between the number
 of data nodes and master eligible nodes in a cluster. I have three master
 eligible nodes and set minimum_master_nodes to 3, and that's it. After that
 I can start or stop additional data nodes as much as I like.

 Jörg

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoG8Yi8KuG7%2BYa3HQEAj%2BDhPUvzoqd6aR-Gpd4iQxAF%3D%2Bw%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoibQk_0o313aXEwYkbYrQiVnmgrp4eNnOMTurmCS6PPw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-15 Thread joergpra...@gmail.com
Using quorum consensus (another name for the 'minimum_master_node'
approach) as default is not possible, since the quorum count is only known
by the admin.

There are perfect solutions for consensus but they are not easy to
implement, see Byzantine fault tolerance

http://en.wikipedia.org/wiki/Byzantine_fault_tolerance

or Paxos

http://research.google.com/archive/paxos_made_live.html

or a more promising approach started lately, RAFT

http://raftconsensus.github.io/

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH4FFgKVeJY37aRuurAt7To5jLa%3D0Rp3b--AcGE%3DhEqoA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread David Pilato
Default value for minimum master node is 1.
No formula here.

It was just an advice. Sorry if I was unclear. :-)

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 14 janv. 2014 à 20:58, Mohit Anchlia mohitanch...@gmail.com a écrit :

 I didn't quite understand. What I am trying to find is that when default 
 formula of N / 2 + 1 I will always have more master nodes on one side 
 correct? And is my understanding incorrect that the other side that has less 
 master nodes than N /2 + 1 would simply not accept any requests? The only 
 point of increasing this value would be for performance reasons?
 
 On Mon, Jan 13, 2014 at 3:58 PM, Mark Walkom ma...@campaignmonitor.com 
 wrote:
 That depends on however many nodes lose contact, but it's a possibility that 
 if you lose 2 of your nodes at once they will just stop responding and the 
 other 3 will be ok.
 However you maintain cluster integrity.
 
 Regards,
 Mark Walkom
 
 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com
 
 
 On 14 January 2014 10:55, Mohit Anchlia mohitanch...@gmail.com wrote:
 How would the nodes on the other side of the network become master with the 
 formula (total num of nodes / 2 + 1) one side will always become 
 inaccessible?
 
 On Mon, Jan 13, 2014 at 3:50 PM, Mark Walkom ma...@campaignmonitor.com 
 wrote:
 The logs shows that a node was not contactable and so was removed from the 
 cluster.
 That could be a load issue on the node, network issues, or something else. 
 Check the logs on the node that disconnected and see what you find.
 
 If you leave the defaults, you risk the chance of a split-brain situation, 
 where nodes lose contact with each other and then decide to become a 
 master. This can cause you to end up with multiple masters in multiple 
 clusters, which is not good for data integrity.
 
 Setting the min masters means that if a node cannot contact N masters, it 
 will refuse connections.
 
 Regards,
 Mark Walkom
 
 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com
 
 
 On 14 January 2014 10:46, Mohit Anchlia mohitanch...@gmail.com wrote:
 I am trying to understand why I should change it? This setting looks like 
 the right setting in most cases. Did you see anything out of ordinary in 
 that log message I pasted?
 
 On Mon, Jan 13, 2014 at 3:40 PM, David Pilato da...@pilato.fr wrote:
 No it's not the default setting.
 min master node = total num of nodes / 2 + 1
 
 You have 5 nodes, 5 / 2 + 1 = 3
 
 
 
 -- 
 David Pilato | Technical Advocate | Elasticsearch.com
 @dadoonet | @elasticsearchfr
 
 
 Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanch...@gmail.com) a 
 écrit:
 
 Is the default setting 3? How do I go about selecting right 
 configuration setting for this property?
 
 On Mon, Jan 13, 2014 at 3:01 PM, David Pilato da...@pilato.fr wrote:
 So I think you should first set minimum master node to 3. See 
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery
 
 Probably you need to close your isolated node and restart it.
 
 
 
 -- 
 David Pilato | Technical Advocate | Elasticsearch.com
 @dadoonet | @elasticsearchfr
 
 
 Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanch...@gmail.com) 
 a écrit:
 
 I have 5 nodes. I am using all the default settings.
 
 On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:
 It sounds like a split brain issue or something.
 
 How many nodes do you have?
 Did you set minimum master node?
 
 -- 
 David Pilato | Technical Advocate | Elasticsearch.com
 @dadoonet | @elasticsearchfr
 
 
 Le 13 janvier 2014 at 22:48:32, Mohit Anchlia 
 (mohitanch...@gmail.com) a écrit:
 
 Thanks! It looks like one of the nodes even though it was up still 
 it went out of cluster. Is there no retry?
  
 [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] 
 master {new 
 [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
  previous 
 [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
  removed 
 {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
  reason: zen-disco-master_failed 
 ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})
 
 [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] 
 removed 
 {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
  reason: 
 zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
  reason failed to ping, tried [3] times, each with maximum [30s] 
 timeout
 
 
 
 On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr 
 wrote:
 Yellow means that some replicas are not allocated.
 If you have only one node and have index set with one replica 
 (default), yellow is the expected state.
 
 
 -- 
 David Pilato | Technical Advocate | 

Re: Cluster state yellow

2014-01-14 Thread Mohit Anchlia
What's the problem we face when ES node starts up? Can you please elaborate?

On Tue, Jan 14, 2014 at 12:54 PM, joergpra...@gmail.com 
joergpra...@gmail.com wrote:

 To be fair, the default minimum_master_node value is -1 (disabled).

 Problem is, you do not know N when an ES node starts up and uses discovery
 - it can grow by time ;)

 Jörg

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAKdsXoED55JCMB65gb-13pNxinqR0F1hDuVvh_dtjLNuyWB1iA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoxQkYnB0cn-rvJG6d6f3q56L8W%3D%2B56akhWZHcNydkw%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread InquiringMind


 *Wouldn't the value increase as you add more nodes?*


Indeed, it most certainly will. 

And with unicast discovery, each node will need to be told about the new 
node. Which is the perfect time to tell it about the newly calculated 
minimum number of masters.
 
Brian

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/54e31afb-ea7f-4854-b970-f534505775cd%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread Mohit Anchlia
But today elasticsearch doesn't do this automatically?

On Tue, Jan 14, 2014 at 2:17 PM, InquiringMind brian.from...@gmail.comwrote:


 *Wouldn't the value increase as you add more nodes?*


 Indeed, it most certainly will.

 And with unicast discovery, each node will need to be told about the new
 node. Which is the perfect time to tell it about the newly calculated
 minimum number of masters.

 Brian

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/54e31afb-ea7f-4854-b970-f534505775cd%40googlegroups.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWq_iK7dFfooQt7HLgZ77WjhZUMV%3DDTtKOSEcU-fWK0-xw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread Mark Walkom
Nope.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 15 January 2014 09:48, Mohit Anchlia mohitanch...@gmail.com wrote:

 But today elasticsearch doesn't do this automatically?

 On Tue, Jan 14, 2014 at 2:17 PM, InquiringMind brian.from...@gmail.comwrote:


 *Wouldn't the value increase as you add more nodes?*


 Indeed, it most certainly will.

 And with unicast discovery, each node will need to be told about the new
 node. Which is the perfect time to tell it about the newly calculated
 minimum number of masters.

 Brian

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/54e31afb-ea7f-4854-b970-f534505775cd%40googlegroups.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWq_iK7dFfooQt7HLgZ77WjhZUMV%3DDTtKOSEcU-fWK0-xw%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624a-47F_w7e0u2jaURfiPiHGL-p5M8-UD1qOe0bMqFKvuA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread Ivan Brusic
Don't forget gateway.expected_nodes :)

Wouldn't the value increase as you add more nodes?

It will, which is precisely why the value is not computed automatically.
The value can decrease/increase over time, but the cluster does not know if
this is because it is on purpose or because of failures. The suggested
(n/2)+1 formula breaks down if n is constantly changing. With 10 nodes, it
is suggested to use the value of 6 (10/2+1) for the minimum number of
nodes. But if 5 nodes crash, then the suggested value is now 3 (5/2+1), but
clearly 3 is not the value we want, we want 6.

Cheers,

Ivan



On Tue, Jan 14, 2014 at 2:58 PM, Mark Walkom ma...@campaignmonitor.comwrote:

 Nope.

 Regards,
 Mark Walkom

 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com


 On 15 January 2014 09:48, Mohit Anchlia mohitanch...@gmail.com wrote:

 But today elasticsearch doesn't do this automatically?

 On Tue, Jan 14, 2014 at 2:17 PM, InquiringMind 
 brian.from...@gmail.comwrote:


 *Wouldn't the value increase as you add more nodes?*


 Indeed, it most certainly will.

 And with unicast discovery, each node will need to be told about the new
 node. Which is the perfect time to tell it about the newly calculated
 minimum number of masters.

 Brian

 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/54e31afb-ea7f-4854-b970-f534505775cd%40googlegroups.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWq_iK7dFfooQt7HLgZ77WjhZUMV%3DDTtKOSEcU-fWK0-xw%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.


  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAEM624a-47F_w7e0u2jaURfiPiHGL-p5M8-UD1qOe0bMqFKvuA%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCMrYAFBv%2BnRCMRYbev0bEaTPHswuUyEBsFvHS8XzKKvA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread InquiringMind


 *But today elasticsearch doesn't do this automatically?*



Short answer: No.
Long answer: Nooo. *:-)*

For unicast discovery, you need to tell each node the full list of nodes in 
the cluster. As far as I can tell, that list may include the local node so 
it's easier to configure the same list everywhere. But it's not automatic.

For multicast discovery, it's automatic. But I've read enough about issues 
with multicast to want to stay with unicast.

Brian

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e2fc5288-9da9-412b-afc6-9e7ba923f7b6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-14 Thread joergpra...@gmail.com
It's not a a matter of unicast vs. multicast or failure vs. non-failure.

It's only the cluster admin who knows what the maximum number of nodes is.

For ES it is impossible to find that out during runtime, and therefore,
minimum_master_nodes is a user defined setting.

Jörg

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHGWAo6A6LK48%2BY0_z2nysrjW_jTud%2Bu1UEtJSJA-aUUA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Cluster state yellow

2014-01-13 Thread Mohit Anchlia
Cluster state is showing yellow. What's the best way to find why it's in
yellow state?

It has been up and stable so far except when I looked today it shows state
turned into yellow.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-13 Thread David Pilato
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow 
is the expected state.


-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow 
state?
 
It has been up and stable so far except when I looked today it shows state 
turned into yellow.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-13 Thread David Pilato
It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out 
of cluster. Is there no retry?
 
[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new 
[10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, 
previous 
[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
{[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed 
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed 
{[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason: 
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout



On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow 
is the expected state.


-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow 
state?
 
It has been up and stable so far except when I looked today it shows state 
turned into yellow.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-13 Thread Mohit Anchlia
I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:

 It sounds like a split brain issue or something.

 How many nodes do you have?
 Did you set minimum master node?

 --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:48:32, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

 Thanks! It looks like one of the nodes even though it was up still it went
 out of cluster. Is there no retry?


 [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
 {new 
 [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
 previous 
 [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
 {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed
 ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
 ]]{group=group2})

 [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed
 {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason:
 zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout


 On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:

  Yellow means that some replicas are not allocated.
  If you have only one node and have index set with one replica (default),
 yellow is the expected state.


  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:30:23, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Cluster state is showing yellow. What's the best way to find why it's
 in yellow state?

 It has been up and stable so far except when I looked today it shows
 state turned into yellow.
  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.


 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com
 .

 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local
 .

 For more options, visit https://groups.google.com/groups/opt_out.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


Re: Cluster state yellow

2014-01-13 Thread Mohit Anchlia
Is the default setting 3? How do I go about selecting right configuration
setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato da...@pilato.fr wrote:

 So I think you should first set minimum master node to 3. See
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

 Probably you need to close your isolated node and restart it.



 --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 23:58:33, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

 I have 5 nodes. I am using all the default settings.

 On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:

  It sounds like a split brain issue or something.

  How many nodes do you have?
  Did you set minimum master node?

  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:48:32, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Thanks! It looks like one of the nodes even though it was up still it
 went out of cluster. Is there no retry?


 [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
 {new 
 [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
 previous 
 [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
 {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed
 ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
 ]]{group=group2})

 [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed
 {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason:
 zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout


 On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:

  Yellow means that some replicas are not allocated.
  If you have only one node and have index set with one replica
 (default), yellow is the expected state.


  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:30:23, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Cluster state is showing yellow. What's the best way to find why it's
 in yellow state?

 It has been up and stable so far except when I looked today it shows
 state turned into yellow.
  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.


 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
  To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.


 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.


 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 

Re: Cluster state yellow

2014-01-13 Thread David Pilato
No it's not the default setting.
min master node = total num of nodes / 2 + 1

You have 5 nodes, 5 / 2 + 1 = 3



-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 14 janvier 2014 at 00:06:34, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Is the default setting 3? How do I go about selecting right configuration 
setting for this property?

On Mon, Jan 13, 2014 at 3:01 PM, David Pilato da...@pilato.fr wrote:
So I think you should first set minimum master node to 3. See 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

Probably you need to close your isolated node and restart it.



-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 23:58:33, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

I have 5 nodes. I am using all the default settings.

On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:
It sounds like a split brain issue or something.

How many nodes do you have?
Did you set minimum master node?

-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 22:48:32, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Thanks! It looks like one of the nodes even though it was up still it went out 
of cluster. Is there no retry?
 
[2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master {new 
[10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1}, 
previous 
[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
{[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed 
([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2})

[2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14] removed 
{[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason: 
zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout



On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:
Yellow means that some replicas are not allocated.
If you have only one node and have index set with one replica (default), yellow 
is the expected state.


-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 13 janvier 2014 at 22:30:23, Mohit Anchlia (mohitanch...@gmail.com) a écrit:

Cluster state is showing yellow. What's the best way to find why it's in yellow 
state?
 
It has been up and stable so far except when I looked today it shows state 
turned into yellow.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.

For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.52d46968.5577f8e1.376c%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrsU-k9u6hir2CWoobhnaOTD_3Bjk3Xf_ZjMDSBkB0DeQ%40mail.gmail.com.

For more options, visit 

Re: Cluster state yellow

2014-01-13 Thread Mohit Anchlia
I am trying to understand why I should change it? This setting looks like
the right setting in most cases. Did you see anything out of ordinary in
that log message I pasted?

On Mon, Jan 13, 2014 at 3:40 PM, David Pilato da...@pilato.fr wrote:

 No it's not the default setting.
 min master node = total num of nodes / 2 + 1

 You have 5 nodes, 5 / 2 + 1 = 3



 --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 14 janvier 2014 at 00:06:34, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

 Is the default setting 3? How do I go about selecting right configuration
 setting for this property?

 On Mon, Jan 13, 2014 at 3:01 PM, David Pilato da...@pilato.fr wrote:

  So I think you should first set minimum master node to 3. See
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

  Probably you need to close your isolated node and restart it.



  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 23:58:33, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   I have 5 nodes. I am using all the default settings.

 On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:

  It sounds like a split brain issue or something.

  How many nodes do you have?
  Did you set minimum master node?

  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:48:32, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Thanks! It looks like one of the nodes even though it was up still it
 went out of cluster. Is there no retry?


 [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14] master
 {new 
 [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
 previous 
 [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
 {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed
 ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
 ]]{group=group2})

 [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
 removed 
 {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason:
 zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout


 On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:

  Yellow means that some replicas are not allocated.
  If you have only one node and have index set with one replica
 (default), yellow is the expected state.


  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:30:23, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Cluster state is showing yellow. What's the best way to find why
 it's in yellow state?

 It has been up and stable so far except when I looked today it shows
 state turned into yellow.
  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/etPan.52d45be9.7c3dbd3d.376c%40MacBook-Air-de-David.local.


 For more options, visit https://groups.google.com/groups/opt_out.


 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
  To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpLn_AgMKcTwQuGD-CHHe6ptg%3DRxaNGLPxFn3kroed%3DVQ%40mail.gmail.com.


 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group 

Re: Cluster state yellow

2014-01-13 Thread Mark Walkom
The logs shows that a node was not contactable and so was removed from the
cluster.
That could be a load issue on the node, network issues, or something else.
Check the logs on the node that disconnected and see what you find.

If you leave the defaults, you risk the chance of a split-brain situation,
where nodes lose contact with each other and then decide to become a
master. This can cause you to end up with multiple masters in multiple
clusters, which is not good for data integrity.

Setting the min masters means that if a node cannot contact N masters, it
will refuse connections.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 14 January 2014 10:46, Mohit Anchlia mohitanch...@gmail.com wrote:

 I am trying to understand why I should change it? This setting looks like
 the right setting in most cases. Did you see anything out of ordinary in
 that log message I pasted?

 On Mon, Jan 13, 2014 at 3:40 PM, David Pilato da...@pilato.fr wrote:

 No it's not the default setting.
 min master node = total num of nodes / 2 + 1

 You have 5 nodes, 5 / 2 + 1 = 3



  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 14 janvier 2014 at 00:06:34, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

 Is the default setting 3? How do I go about selecting right configuration
 setting for this property?

 On Mon, Jan 13, 2014 at 3:01 PM, David Pilato da...@pilato.fr wrote:

  So I think you should first set minimum master node to 3. See
 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_discovery

  Probably you need to close your isolated node and restart it.



  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 23:58:33, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   I have 5 nodes. I am using all the default settings.

 On Mon, Jan 13, 2014 at 2:32 PM, David Pilato da...@pilato.fr wrote:

  It sounds like a split brain issue or something.

  How many nodes do you have?
  Did you set minimum master node?

  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:48:32, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Thanks! It looks like one of the nodes even though it was up still
 it went out of cluster. Is there no retry?


 [2014-01-02 17:55:01,148][INFO ][cluster.service ] [10.80.140.14]
 master {new 
 [10.80.140.14][267foffDRhO6zzXlHg3YKA][inet[/10.80.140.14:9300]]{group=group1},
 previous 
 [10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2}},
 removed 
 {[10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300]]{group=group2},},
 reason: zen-disco-master_failed
 ([10.80.140.59][YDHawZWnRUKukXDIIT5MJQ][inet[/10.80.140.59:9300
 ]]{group=group2})

 [2014-01-02 17:55:02,158][INFO ][cluster.service ] [10.80.140.14]
 removed 
 {[10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2},},
 reason:
 zen-disco-node_failed([10.80.140.13][6cf39fcASKSn6dNNsSXguA][inet[/10.80.140.13:9300]]{group=group2}),
 reason failed to ping, tried [3] times, each with maximum [30s] timeout


 On Mon, Jan 13, 2014 at 1:34 PM, David Pilato da...@pilato.fr wrote:

  Yellow means that some replicas are not allocated.
  If you have only one node and have index set with one replica
 (default), yellow is the expected state.


  --
 *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
 @dadoonet https://twitter.com/dadoonet | 
 @elasticsearchfrhttps://twitter.com/elasticsearchfr


 Le 13 janvier 2014 at 22:30:23, Mohit Anchlia 
 (mohitanch...@gmail.com//mohitanch...@gmail.com)
 a écrit:

   Cluster state is showing yellow. What's the best way to find why
 it's in yellow state?

 It has been up and stable so far except when I looked today it shows
 state turned into yellow.
  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpSSEVTpYA-DB8Hgfyo9WdR4bVRoKtQ9c_PC%2BFZ1aQiKg%40mail.gmail.com
 .
 For more options, visit https://groups.google.com/groups/opt_out.

  --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion