No, I use unicast discovery, not the Azure plugin. From: Boaz Leskes <b.les...@gmail.com<mailto:b.les...@gmail.com>> Reply-To: "elasticsearch@googlegroups.com<mailto:elasticsearch@googlegroups.com>" <elasticsearch@googlegroups.com<mailto:elasticsearch@googlegroups.com>> Date: Monday, November 10, 2014 at 10:52 PM To: "elasticsearch@googlegroups.com<mailto:elasticsearch@googlegroups.com>" <elasticsearch@googlegroups.com<mailto:elasticsearch@googlegroups.com>> Subject: Re: Nodes not joining after 1.4.0 upgrade
One more thing - do you use the Azure plugin for ES? On Tuesday, November 11, 2014 7:50:47 AM UTC+1, Boaz Leskes wrote: Hi Janet, Was there anything in the master logs, i.e., gnslogstash10 in your example? Cheers, Boaz On Monday, November 10, 2014 11:47:22 PM UTC+1, Janet Sullivan wrote: I’m also using unicast discovery, as multicast doesn’t work on Azure. I ended up in a bad position - 1.4 wouldn’t come up all the way, but 1.3.4 wouldn’t accept shards with the new lucene version. I ended up rebuilding the cluster, and I’m going to have to backfill from text logs. A fresh 1.4 cluster works fine, but after two days I couldn’t get the upgraded cluster to work. I’m glad to hear someone else had a similar issue. On Nov 10, 2014, at 7:05 AM, Valentin <plet...@gmail.com<mailto:plet...@gmail.com>> wrote: I had similar issues when upgrading from 1.3.4 to 1.4 from my elasticsearch.yml discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts:..... I could get it up and running after restarting the whole cluster (which was bad since I'm using it for realtime logging). On Monday, November 10, 2014 1:34:12 PM UTC+1, Boaz Leskes wrote: Hi, The logs you mentioned indicate that the nodes try to join the cluster but it takes too long for a complete verification cycle (connect back to node and publish cluster state to it) takes too long. It seems there is something going on your masters. Can you check the logs over there? Also are you using multicast or unicast discovery? On Sunday, November 9, 2014 8:36:06 AM UTC+1, Janet Sullivan wrote: More hours of working – even when I get a 1.4.0 cluster up, masters wouldn’t fail over – when I took master1 down, neither master2 or master3 would promote themselves. In 1.4.0-beta it fails over quickly. From: elasti...@googlegroups.com [mailto:elasti...@googlegroups.com] On Behalf Of Janet Sullivan Sent: Saturday, November 08, 2014 11:11 PM To: elasti...@googlegroups.com Subject: RE: Nodes not joining after 1.4.0 upgrade OK, it also happens to some degree with 1.4.0-beta, although overall it’s much better on beta. I wasn’t able to get my 12 node cluster up on 1.4.0 after several hours of fiddling, but 1.4.0-beta did come up. From: elasti...@googlegroups.com [mailto:ela...@googlegroups.com] On Behalf Of Janet Sullivan Sent: Saturday, November 08, 2014 9:26 PM To: elasti...@googlegroups.com Subject: RE: Nodes not joining after 1.4.0 upgrade But it DOES happen with 1.3.5. Hmmm…. From: elasti...@googlegroups.com [mailto:ela...@googlegroups.com] On Behalf Of Janet Sullivan Sent: Saturday, November 08, 2014 9:24 PM To: elasti...@googlegroups.com Subject: RE: Nodes not joining after 1.4.0 upgrade Note: This issue doesn’t happen with 1.4.0-beta1 From: elasti...@googlegroups.com [mailto:ela...@googlegroups.com] On Behalf Of Janet Sullivan Sent: Saturday, November 08, 2014 8:46 PM To: elasti...@googlegroups.com Subject: Nodes not joining after 1.4.0 upgrade I’ve upgraded a couple of clusters to 1.4.0 from 1.3.4. On both of them, I had nodes that spewed the following, and were slow to join, if they joined at all: [2014-11-09 04:33:45,995][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:34:49,776][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:35:53,571][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:36:57,353][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:38:01,120][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:39:04,885][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] [2014-11-09 04:40:08,657][INFO ][discovery.zen ] [gnslogstash3] failed to send join request to master [[gnslogstash10][9nx_f_NiQtCntgnD2q7k0g][gnslogstash10][inet[/10.0.0.29:9300]]{data=false, master=true}], reason [ElasticsearchTimeoutException[Timeout waiting for task.]] I’m able to telnet to port 9300 on gnslogstash10 in this example from gnslogstash3 with no issue, but this cluster doesn’t want to bring all its nodes up. The more nodes added, the more likely a join will fail. In this example, 9 nodes are up, but 3 nodes don’t want to join. :( Thoughts? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2817051de55a4f96a4bd173f26177f29%40CY1PR0701MB1164.namprd07.prod.outlook.com<https://groups.google.com/d/msgid/elasticsearch/2817051de55a4f96a4bd173f26177f29%40CY1PR0701MB1164.namprd07.prod.outlook.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2098c745e47648d6b668b8c699f8750c%40CY1PR0701MB1164.namprd07.prod.outlook.com<https://groups.google.com/d/msgid/elasticsearch/2098c745e47648d6b668b8c699f8750c%40CY1PR0701MB1164.namprd07.prod.outlook.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/47110bba4ae6400c9888beb70972c6f0%40CY1PR0701MB1164.namprd07.prod.outlook.com<https://groups.google.com/d/msgid/elasticsearch/47110bba4ae6400c9888beb70972c6f0%40CY1PR0701MB1164.namprd07.prod.outlook.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a296bdb68984733af72024cfe8657f6%40CY1PR0701MB1164.namprd07.prod.outlook.com<https://groups.google.com/d/msgid/elasticsearch/2a296bdb68984733af72024cfe8657f6%40CY1PR0701MB1164.namprd07.prod.outlook.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com<mailto:elasticsearch+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/02354045-84fb-4820-a9ea-8b06279cead9%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/02354045-84fb-4820-a9ea-8b06279cead9%40googlegroups.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com<mailto:elasticsearch+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/86bb1cde-be20-43bc-a832-8364603174cc%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/86bb1cde-be20-43bc-a832-8364603174cc%40googlegroups.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D087024A.19055%25janets%40nairial.net. For more options, visit https://groups.google.com/d/optout.
