RE: 1.4.0 data node can't join existing 1.3.4 cluster

[Christian Hedegaard]

This is working perfectly! I’ve got a test cluster that I’m in the middle of 
doing a rolling restart of with no issues:

elasticsearch-"number" : "1.4.0",
elasticsearch- "number" : "1.4.0",
elasticsearch- "number" : "1.3.4",
elasticsearch- "number" : "1.3.4",
elasticsearch-"number" : "1.3.4",

_cluster/health:

  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 5,
  "active_primary_shards" : 1730,
  "active_shards" : 3460,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0

I anticipate no other problems finishing this rolling upgrade. Thanks a ton 
everyone!


From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On 
Behalf Of David Pilato
Sent: Monday, November 24, 2014 8:31 AM
To: elasticsearch@googlegroups.com
Subject: Re: 1.4.0 data node can't join existing 1.3.4 cluster

Heya,

We will release aws plugin 2.4.1 in some minutes.
It fixes this rolling upgrade issue.

Note that some WARN messages could appear in old nodes logs until the full 
rolling upgrade is done.

Thank you all for reporting this!


Le dimanche 23 novembre 2014 03:10:42 UTC+1, Ivan Brusic a écrit :

Great work everyone. Feel better about upgrading now.
On Nov 22, 2014 4:42 PM, "Boaz Leskes" 
mailto:b.les...@gmail.com>> wrote:
Hi Christian, Daniel,

I believe I found the issue - it has to do with the cloud plugins (both AWS and 
GCE) and the way they create the node list for the unicast based discovery. 
Effectively they mislead it to think that that all nodes on the cluster are 
version 1.4.0 which is not correct.

I opened issues for this so it will be corrected soon: 
https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143 , 
https://github.com/elasticsearch/elasticsearch-cloud-gce/issues/41

Cheers,
Boaz


On Saturday, November 22, 2014 7:04:33 PM UTC+2, Jörg Prante wrote:
As said, the change is due to unicast action, which was split in 1.4.0 to an 
old and a new action, see this commit:

https://github.com/elasticsearch/elasticsearch/commit/e5de47d928582694c7729d199390086983779e6e<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>

I am not sure if this is a bug. It seems like a feature to prevent multiple 
masters by accident.

The strategy as described above by Christian Hedegaard should work, it is still 
to be considered a work-around:

- setting up all new 1.4 nodes as not master eligible ("data only")

- joining them to a 1.3.x cluster while master still is on a 1.3 node should 
work

- then, shutting down all 1.3 nodes (except the master) should relocate the 
shards

- bringing down the final 1.3 master should "stall" master election (I would 
also configure a large timeout for master election). This is critical, no 
index/mapping creations/deletions or cluster state modifying actions should be 
executed now.

- adding a 1.4 master eligible node should now overtake the cluster (I would 
start it with the data folder from the final 1.3 master where the last cluster 
state is persisted) and the critical phase is over.

- from then, more 1.4 master eligible nodes should be possible to add

- finally, the minimum master nodes setting should be configured

Jörg


On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard 
mailto:chedega...@red5studios.com>> wrote:
FYI, I have found a solution that works (at least for me).

I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve done is 
bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I added a line 
to point the nodes via unicast explicitly to the current master:
discovery.zen.ping.unicast.hosts: 
["10.210.9.224:9300<http://10.210.9.224:9300>"]

When I restarted elasticsearch with that setting, with cloud-aws installed and 
configured on version 2.4.0, the new nodes found the cluster and properly 
joined it.

I will now start nuking the old v1.3.5 nodes to migrate the data off of them. 
Before the final 1.3.5 node is nuked, I will change the config on one of the 
v1.4.0 nodes to allow it as master and restart it.

I’m not sure if the master stuff is needed or not, but I was very afraid of a 
split-brain problem. I have another 4-node testing cluster that I will be able 
to try this upgrade again with in a more controlled manner.

I’m NOT looking forward to upgrading our current production cluster this way 
(15 data-only nodes, 3 master-only nodes).

So it would appear that the problem is somewhere in the unicast discovery code. 
The question is who’s to b

RE: 1.4.0 data node can't join existing 1.3.4 cluster

[Christian Hedegaard]

Awesome! I’ll monitor the cloud-aws plugins github. Once they get a fix I can 
test it out on another one of our testing clusters.

From: elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] On 
Behalf Of Boaz Leskes
Sent: Saturday, November 22, 2014 1:43 PM
To: elasticsearch@googlegroups.com
Subject: Re: 1.4.0 data node can't join existing 1.3.4 cluster

Hi Christian, Daniel,

I believe I found the issue - it has to do with the cloud plugins (both AWS and 
GCE) and the way they create the node list for the unicast based discovery. 
Effectively they mislead it to think that that all nodes on the cluster are 
version 1.4.0 which is not correct.

I opened issues for this so it will be corrected soon: 
https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143 , 
https://github.com/elasticsearch/elasticsearch-cloud-gce/issues/41

Cheers,
Boaz


On Saturday, November 22, 2014 7:04:33 PM UTC+2, Jörg Prante wrote:
As said, the change is due to unicast action, which was split in 1.4.0 to an 
old and a new action, see this commit:

https://github.com/elasticsearch/elasticsearch/commit/e5de47d928582694c7729d199390086983779e6e<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>

I am not sure if this is a bug. It seems like a feature to prevent multiple 
masters by accident.

The strategy as described above by Christian Hedegaard should work, it is still 
to be considered a work-around:

- setting up all new 1.4 nodes as not master eligible ("data only")

- joining them to a 1.3.x cluster while master still is on a 1.3 node should 
work

- then, shutting down all 1.3 nodes (except the master) should relocate the 
shards

- bringing down the final 1.3 master should "stall" master election (I would 
also configure a large timeout for master election). This is critical, no 
index/mapping creations/deletions or cluster state modifying actions should be 
executed now.

- adding a 1.4 master eligible node should now overtake the cluster (I would 
start it with the data folder from the final 1.3 master where the last cluster 
state is persisted) and the critical phase is over.

- from then, more 1.4 master eligible nodes should be possible to add

- finally, the minimum master nodes setting should be configured

Jörg


On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard 
mailto:chedega...@red5studios.com>> wrote:
FYI, I have found a solution that works (at least for me).

I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve done is 
bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I added a line 
to point the nodes via unicast explicitly to the current master:
discovery.zen.ping.unicast.hosts: 
["10.210.9.224:9300<http://10.210.9.224:9300>"]

When I restarted elasticsearch with that setting, with cloud-aws installed and 
configured on version 2.4.0, the new nodes found the cluster and properly 
joined it.

I will now start nuking the old v1.3.5 nodes to migrate the data off of them. 
Before the final 1.3.5 node is nuked, I will change the config on one of the 
v1.4.0 nodes to allow it as master and restart it.

I’m not sure if the master stuff is needed or not, but I was very afraid of a 
split-brain problem. I have another 4-node testing cluster that I will be able 
to try this upgrade again with in a more controlled manner.

I’m NOT looking forward to upgrading our current production cluster this way 
(15 data-only nodes, 3 master-only nodes).

So it would appear that the problem is somewhere in the unicast discovery code. 
The question is who’s to blame? Elasticsearch or the cloud-aws plugin?



From: Boaz Leskes [mailto:b.les...@gmail.com<mailto:b.les...@gmail.com>]
Sent: Wednesday, November 19, 2014 2:27 PM
To: elasticsearch@googlegroups.com<mailto:elasticsearch@googlegroups.com>
Cc: Christian Hedegaard
Subject: Re: 1.4.0 data node can't join existing 1.3.4 cluster

Hi Christian,

I'm not sure what thread you refer to exactly, but this shouldn't happen. Can 
you describe the problem you have some more? Anything in the nodes? (both the 
1.4 node and the master)

Cheers,
Boaz

On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard wrote:
I found this thread while trying to research the same issue and it looks like 
there is currently no resolution. We like to keep up on our elasticsearch 
upgrades as often as possible and do rolling upgrades to keep our clusters up. 
When testing I’m having the same issue, I cannot add a 1.4.0 box to the 
existing 1.3.4 cluster.

Is there a fix for this anticipated?
--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
elasticsearch+unsubscr...@g

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[David Pilato]

Heya,

We will release aws plugin 2.4.1 in some minutes.
It fixes this rolling upgrade issue.

Note that some WARN messages could appear in old nodes logs until the full 
rolling upgrade is done.

Thank you all for reporting this!


Le dimanche 23 novembre 2014 03:10:42 UTC+1, Ivan Brusic a écrit :
>
> Great work everyone. Feel better about upgrading now.
>  On Nov 22, 2014 4:42 PM, "Boaz Leskes"  wrote:
>
>> Hi Christian, Daniel,
>>
>> I believe I found the issue - it has to do with the cloud plugins (both 
>> AWS and GCE) and the way they create the node list for the unicast based 
>> discovery. Effectively they mislead it to think that that all nodes on the 
>> cluster are version 1.4.0 which is not correct. 
>>
>> I opened issues for this so it will be corrected soon: 
>> https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143 , 
>> https://github.com/elasticsearch/elasticsearch-cloud-gce/issues/41
>>
>> Cheers,
>> Boaz
>>
>>
>> On Saturday, November 22, 2014 7:04:33 PM UTC+2, Jörg Prante wrote:
>>>
>>> As said, the change is due to unicast action, which was split in 1.4.0 
>>> to an old and a new action, see this commit:
>>>
>>> https://github.com/elasticsearch/elasticsearch/commit/
>>> e5de47d928582694c7729d199390086983779e6e 
>>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>
>>>
>>> I am not sure if this is a bug. It seems like a feature to prevent 
>>> multiple masters by accident.
>>>
>>> The strategy as described above by Christian Hedegaard should work, it 
>>> is still to be considered a work-around:
>>>
>>> - setting up all new 1.4 nodes as not master eligible ("data only")
>>>
>>> - joining them to a 1.3.x cluster while master still is on a 1.3 node 
>>> should work
>>>
>>> - then, shutting down all 1.3 nodes (except the master) should relocate 
>>> the shards
>>>
>>> - bringing down the final 1.3 master should "stall" master election (I 
>>> would also configure a large timeout for master election). This is 
>>> critical, no index/mapping creations/deletions or cluster state modifying 
>>> actions should be executed now.
>>>
>>> - adding a 1.4 master eligible node should now overtake the cluster (I 
>>> would start it with the data folder from the final 1.3 master where the 
>>> last cluster state is persisted) and the critical phase is over.
>>>
>>> - from then, more 1.4 master eligible nodes should be possible to add
>>>
>>> - finally, the minimum master nodes setting should be configured
>>>
>>> Jörg
>>>
>>>
>>> On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard <
>>> chedega...@red5studios.com> wrote:
>>>
>>>>  FYI, I have found a solution that works (at least for me).
>>>>
>>>>  
>>>>
>>>> I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve 
>>>> done is bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I 
>>>> added a line to point the nodes via unicast explicitly to the current 
>>>> master:
>>>>
>>>> discovery.zen.ping.unicast.hosts: ["10.210.9.224:9300"]
>>>>
>>>>  
>>>>
>>>> When I restarted elasticsearch with that setting, with cloud-aws 
>>>> installed and configured on version 2.4.0, the new nodes found the cluster 
>>>> and properly joined it.
>>>>
>>>>  
>>>>
>>>> I will now start nuking the old v1.3.5 nodes to migrate the data off of 
>>>> them. Before the final 1.3.5 node is nuked, I will change the config on 
>>>> one 
>>>> of the v1.4.0 nodes to allow it as master and restart it.
>>>>
>>>>  
>>>>
>>>> I’m not sure if the master stuff is needed or not, but I was very 
>>>> afraid of a split-brain problem. I have another 4-node testing cluster 
>>>> that 
>>>> I will be able to try this upgrade again with in a more controlled manner.
>>>>
>>>>  
>>>>
>>>> I’m NOT looking forward to upgrading our current production cluster 
>>>> this way (15 data-only nodes, 3 master-only nodes).
>>>>
>>>>  
>>>>
>>>&

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Ivan Brusic]

Great work everyone. Feel better about upgrading now.
 On Nov 22, 2014 4:42 PM, "Boaz Leskes"  wrote:

> Hi Christian, Daniel,
>
> I believe I found the issue - it has to do with the cloud plugins (both
> AWS and GCE) and the way they create the node list for the unicast based
> discovery. Effectively they mislead it to think that that all nodes on the
> cluster are version 1.4.0 which is not correct.
>
> I opened issues for this so it will be corrected soon:
> https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143 ,
> https://github.com/elasticsearch/elasticsearch-cloud-gce/issues/41
>
> Cheers,
> Boaz
>
>
> On Saturday, November 22, 2014 7:04:33 PM UTC+2, Jörg Prante wrote:
>>
>> As said, the change is due to unicast action, which was split in 1.4.0 to
>> an old and a new action, see this commit:
>>
>> https://github.com/elasticsearch/elasticsearch/commit/
>> e5de47d928582694c7729d199390086983779e6e
>> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>
>>
>> I am not sure if this is a bug. It seems like a feature to prevent
>> multiple masters by accident.
>>
>> The strategy as described above by Christian Hedegaard should work, it is
>> still to be considered a work-around:
>>
>> - setting up all new 1.4 nodes as not master eligible ("data only")
>>
>> - joining them to a 1.3.x cluster while master still is on a 1.3 node
>> should work
>>
>> - then, shutting down all 1.3 nodes (except the master) should relocate
>> the shards
>>
>> - bringing down the final 1.3 master should "stall" master election (I
>> would also configure a large timeout for master election). This is
>> critical, no index/mapping creations/deletions or cluster state modifying
>> actions should be executed now.
>>
>> - adding a 1.4 master eligible node should now overtake the cluster (I
>> would start it with the data folder from the final 1.3 master where the
>> last cluster state is persisted) and the critical phase is over.
>>
>> - from then, more 1.4 master eligible nodes should be possible to add
>>
>> - finally, the minimum master nodes setting should be configured
>>
>> Jörg
>>
>>
>> On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard <
>> chedega...@red5studios.com> wrote:
>>
>>>  FYI, I have found a solution that works (at least for me).
>>>
>>>
>>>
>>> I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve
>>> done is bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I
>>> added a line to point the nodes via unicast explicitly to the current
>>> master:
>>>
>>> discovery.zen.ping.unicast.hosts: ["10.210.9.224:9300"]
>>>
>>>
>>>
>>> When I restarted elasticsearch with that setting, with cloud-aws
>>> installed and configured on version 2.4.0, the new nodes found the cluster
>>> and properly joined it.
>>>
>>>
>>>
>>> I will now start nuking the old v1.3.5 nodes to migrate the data off of
>>> them. Before the final 1.3.5 node is nuked, I will change the config on one
>>> of the v1.4.0 nodes to allow it as master and restart it.
>>>
>>>
>>>
>>> I’m not sure if the master stuff is needed or not, but I was very afraid
>>> of a split-brain problem. I have another 4-node testing cluster that I will
>>> be able to try this upgrade again with in a more controlled manner.
>>>
>>>
>>>
>>> I’m NOT looking forward to upgrading our current production cluster this
>>> way (15 data-only nodes, 3 master-only nodes).
>>>
>>>
>>>
>>> So it would appear that the problem is somewhere in the unicast
>>> discovery code. The question is who’s to blame? Elasticsearch or the
>>> cloud-aws plugin?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *From:* Boaz Leskes [mailto:b.les...@gmail.com]
>>> *Sent:* Wednesday, November 19, 2014 2:27 PM
>>> *To:* elasticsearch@googlegroups.com
>>> *Cc:* Christian Hedegaard
>>> *Subject:* Re: 1.4.0 data node can't join existing 1.3.4 cluster
>>>
>>>
>>>
>>> Hi Christian,
>>>
>>>
>>>
>>> I'm not sure what thread you refer to exactly, but this shouldn't
>>&g

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Boaz Leskes]

Hi Christian, Daniel,

I believe I found the issue - it has to do with the cloud plugins (both AWS 
and GCE) and the way they create the node list for the unicast based 
discovery. Effectively they mislead it to think that that all nodes on the 
cluster are version 1.4.0 which is not correct. 

I opened issues for this so it will be corrected 
soon: https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143 
, https://github.com/elasticsearch/elasticsearch-cloud-gce/issues/41

Cheers,
Boaz


On Saturday, November 22, 2014 7:04:33 PM UTC+2, Jörg Prante wrote:
>
> As said, the change is due to unicast action, which was split in 1.4.0 to 
> an old and a new action, see this commit:
>
>
> https://github.com/elasticsearch/elasticsearch/commit/e5de47d928582694c7729d199390086983779e6e
>  
> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>
>
> I am not sure if this is a bug. It seems like a feature to prevent 
> multiple masters by accident.
>
> The strategy as described above by Christian Hedegaard should work, it is 
> still to be considered a work-around:
>
> - setting up all new 1.4 nodes as not master eligible ("data only")
>
> - joining them to a 1.3.x cluster while master still is on a 1.3 node 
> should work
>
> - then, shutting down all 1.3 nodes (except the master) should relocate 
> the shards
>
> - bringing down the final 1.3 master should "stall" master election (I 
> would also configure a large timeout for master election). This is 
> critical, no index/mapping creations/deletions or cluster state modifying 
> actions should be executed now.
>
> - adding a 1.4 master eligible node should now overtake the cluster (I 
> would start it with the data folder from the final 1.3 master where the 
> last cluster state is persisted) and the critical phase is over.
>
> - from then, more 1.4 master eligible nodes should be possible to add
>
> - finally, the minimum master nodes setting should be configured
>
> Jörg
>
>
> On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard <
> chedega...@red5studios.com> wrote:
>
>>  FYI, I have found a solution that works (at least for me).
>>
>>  
>>
>> I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve done 
>> is bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I added 
>> a line to point the nodes via unicast explicitly to the current master:
>>
>> discovery.zen.ping.unicast.hosts: ["10.210.9.224:9300"]
>>
>>  
>>
>> When I restarted elasticsearch with that setting, with cloud-aws 
>> installed and configured on version 2.4.0, the new nodes found the cluster 
>> and properly joined it.
>>
>>  
>>
>> I will now start nuking the old v1.3.5 nodes to migrate the data off of 
>> them. Before the final 1.3.5 node is nuked, I will change the config on one 
>> of the v1.4.0 nodes to allow it as master and restart it.
>>
>>  
>>
>> I’m not sure if the master stuff is needed or not, but I was very afraid 
>> of a split-brain problem. I have another 4-node testing cluster that I will 
>> be able to try this upgrade again with in a more controlled manner.
>>
>>  
>>
>> I’m NOT looking forward to upgrading our current production cluster this 
>> way (15 data-only nodes, 3 master-only nodes).
>>
>>  
>>
>> So it would appear that the problem is somewhere in the unicast discovery 
>> code. The question is who’s to blame? Elasticsearch or the cloud-aws plugin?
>>
>>  
>>
>>  
>>
>>  
>>
>> *From:* Boaz Leskes [mailto:b.les...@gmail.com] 
>> *Sent:* Wednesday, November 19, 2014 2:27 PM
>> *To:* elasticsearch@googlegroups.com
>> *Cc:* Christian Hedegaard
>> *Subject:* Re: 1.4.0 data node can't join existing 1.3.4 cluster
>>
>>  
>>  
>> Hi Christian, 
>>  
>>  
>>  
>> I'm not sure what thread you refer to exactly, but this shouldn't happen. 
>> Can you describe the problem you have some more? Anything in the nodes? 
>> (both the 1.4 node and the master)
>>  
>>  
>>  
>> Cheers,
>>  
>> Boaz
>>  
>>
>> On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard 
>> wrote:
>>  
>> I found this thread while trying to research the same issue and it looks 
>> like there is currently no resolution. We like to keep up on our 
>> elasticsearch upgrades as often as possib

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Boaz Leskes]

Hi All,

I believe I found the source of the problem and it has to do with the AWS 
plugin. I opened an issue for it, which should be pretty easy to 
fix: https://github.com/elasticsearch/elasticsearch-cloud-aws/issues/143  .

Cheers,
Boaz

On Friday, November 21, 2014 5:39:32 PM UTC+2, Ivan Brusic wrote:
>
> Has an official issue been created? I would like to track the status.
>
> So far, every 1.x.0 release has been buggy. :)
>
> -- 
> Ivan
>
> On Fri, Nov 21, 2014 at 4:06 AM, Mark Walkom  wrote:
>
>> It's being looked at, but I don't know much beyond that at the moment 
>> sorry.
>>
>> On 21 November 2014 20:02,  wrote:
>>
>>> Is there any of the elasticsearch team members that can hint to whether 
>>> or not this is something that will be fixed in 1.4.1? Then we'll simply 
>>> wait for it instead of doing different hacks to upgrade.
>>>
>>> On Monday, November 17, 2014 12:35:03 PM UTC+1, Matthew Barrington wrote:

 I stand corrected, this did not work on our main cluster.

 On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:
>
> We are running a 1.3.4 cluster using the AWS plugin and I noticed the 
> same error when I tried to upgrade a single node.
>
> Since I was trying this on my test cluster first I decided to see what 
> would happen if I upgraded a 2nd node. Would it split into 2 clusters, 
> have 
> the same issue, etc.
>
> What I discovered was that when 2 nodes were upgraded to 1.4 they 
> joined the cluster correctly and everything looks to be working.
>
> SO the problem seems to be for the initial node to join, but when you 
> try with two everything works out.
>
> On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>>
>> On Fri, Nov 14, 2014 at 3:41 AM,   wrote: 
>> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
>> cluster 
>> > with cloud-aws plugin version 2.4.0. Is there a workaround to use 
>> during 
>> > upgrade, since I assume it's not a problem when they're all 
>> upgraded to 
>> > 1.4.0. 
>>
>> I ended up starting a new cluster (ignoring all the warnings logged 
>> on 
>> startup), and restoring from a snapshot. Once all the 1.3.4 nodes 
>> were 
>> gone, no issues. 
>>
>> -- 
>> Eric Jain 
>> Got data? Get answers at zenobase.com. 
>>
>  -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/312dcdc1-d826-4cb9-b480-620232634ea7%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZn-ryDDoQps-smzUPkJd5ru9EHfKuAGRReU2-J-C35kvA%40mail.gmail.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/fd080846-ce10-4e9d-885a-8ad406e03944%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[joergpra...@gmail.com]

As said, the change is due to unicast action, which was split in 1.4.0 to
an old and a new action, see this commit:

https://github.com/elasticsearch/elasticsearch/commit/e5de47d928582694c7729d199390086983779e6e
<https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Felasticsearch%2Felasticsearch%2Fcommit%2Fe5de47d928582694c7729d199390086983779e6e&sa=D&sntz=1&usg=AFQjCNFQkgiVz8SfE_dZ5Sa5K7TqYCIQ6g>

I am not sure if this is a bug. It seems like a feature to prevent multiple
masters by accident.

The strategy as described above by Christian Hedegaard should work, it is
still to be considered a work-around:

- setting up all new 1.4 nodes as not master eligible ("data only")

- joining them to a 1.3.x cluster while master still is on a 1.3 node
should work

- then, shutting down all 1.3 nodes (except the master) should relocate the
shards

- bringing down the final 1.3 master should "stall" master election (I
would also configure a large timeout for master election). This is
critical, no index/mapping creations/deletions or cluster state modifying
actions should be executed now.

- adding a 1.4 master eligible node should now overtake the cluster (I
would start it with the data folder from the final 1.3 master where the
last cluster state is persisted) and the critical phase is over.

- from then, more 1.4 master eligible nodes should be possible to add

- finally, the minimum master nodes setting should be configured

Jörg


On Fri, Nov 21, 2014 at 1:56 AM, Christian Hedegaard <
chedega...@red5studios.com> wrote:

>  FYI, I have found a solution that works (at least for me).
>
>
>
> I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve done
> is bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I added
> a line to point the nodes via unicast explicitly to the current master:
>
> discovery.zen.ping.unicast.hosts: ["10.210.9.224:9300"]
>
>
>
> When I restarted elasticsearch with that setting, with cloud-aws installed
> and configured on version 2.4.0, the new nodes found the cluster and
> properly joined it.
>
>
>
> I will now start nuking the old v1.3.5 nodes to migrate the data off of
> them. Before the final 1.3.5 node is nuked, I will change the config on one
> of the v1.4.0 nodes to allow it as master and restart it.
>
>
>
> I’m not sure if the master stuff is needed or not, but I was very afraid
> of a split-brain problem. I have another 4-node testing cluster that I will
> be able to try this upgrade again with in a more controlled manner.
>
>
>
> I’m NOT looking forward to upgrading our current production cluster this
> way (15 data-only nodes, 3 master-only nodes).
>
>
>
> So it would appear that the problem is somewhere in the unicast discovery
> code. The question is who’s to blame? Elasticsearch or the cloud-aws plugin?
>
>
>
>
>
>
>
> *From:* Boaz Leskes [mailto:b.les...@gmail.com]
> *Sent:* Wednesday, November 19, 2014 2:27 PM
> *To:* elasticsearch@googlegroups.com
> *Cc:* Christian Hedegaard
> *Subject:* Re: 1.4.0 data node can't join existing 1.3.4 cluster
>
>
>
> Hi Christian,
>
>
>
> I'm not sure what thread you refer to exactly, but this shouldn't happen.
> Can you describe the problem you have some more? Anything in the nodes?
> (both the 1.4 node and the master)
>
>
>
> Cheers,
>
> Boaz
>
>
> On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard
> wrote:
>
> I found this thread while trying to research the same issue and it looks
> like there is currently no resolution. We like to keep up on our
> elasticsearch upgrades as often as possible and do rolling upgrades to keep
> our clusters up. When testing I’m having the same issue, I cannot add a
> 1.4.0 box to the existing 1.3.4 cluster.
>
>
>
> Is there a fix for this anticipated?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/5CF8216AA982AF47A8E6DEACA629D22B4EBF409B%40s-us-ex-6.US.R5S.com
> <https://groups.google.com/d/msgid/elasticsearch/5CF8216AA982AF47A8E6DEACA629D22B4EBF409B%40s-us-ex-6.US.R5S.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFYYBDQBXVhr-egAspoY_sy6PqW1wBh%3DPAEGM%2B1%2Bab%2Bew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Ivan Brusic]

Has an official issue been created? I would like to track the status.

So far, every 1.x.0 release has been buggy. :)

-- 
Ivan

On Fri, Nov 21, 2014 at 4:06 AM, Mark Walkom  wrote:

> It's being looked at, but I don't know much beyond that at the moment
> sorry.
>
> On 21 November 2014 20:02,  wrote:
>
>> Is there any of the elasticsearch team members that can hint to whether
>> or not this is something that will be fixed in 1.4.1? Then we'll simply
>> wait for it instead of doing different hacks to upgrade.
>>
>> On Monday, November 17, 2014 12:35:03 PM UTC+1, Matthew Barrington wrote:
>>>
>>> I stand corrected, this did not work on our main cluster.
>>>
>>> On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:

 We are running a 1.3.4 cluster using the AWS plugin and I noticed the
 same error when I tried to upgrade a single node.

 Since I was trying this on my test cluster first I decided to see what
 would happen if I upgraded a 2nd node. Would it split into 2 clusters, have
 the same issue, etc.

 What I discovered was that when 2 nodes were upgraded to 1.4 they
 joined the cluster correctly and everything looks to be working.

 SO the problem seems to be for the initial node to join, but when you
 try with two everything works out.

 On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>
> On Fri, Nov 14, 2014 at 3:41 AM,   wrote:
> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4
> cluster
> > with cloud-aws plugin version 2.4.0. Is there a workaround to use
> during
> > upgrade, since I assume it's not a problem when they're all upgraded
> to
> > 1.4.0.
>
> I ended up starting a new cluster (ignoring all the warnings logged on
> startup), and restoring from a snapshot. Once all the 1.3.4 nodes were
> gone, no issues.
>
> --
> Eric Jain
> Got data? Get answers at zenobase.com.
>
  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/312dcdc1-d826-4cb9-b480-620232634ea7%40googlegroups.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZn-ryDDoQps-smzUPkJd5ru9EHfKuAGRReU2-J-C35kvA%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDZ8k8GQQJn89V4W4S1Bm%3DDKfgMBsaB6a%2B4TcFr67nJkg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[madsmartin]

Cool :)

Usually this means a fix will emerge. Thanks!

On Friday, November 21, 2014 10:07:03 AM UTC+1, Mark Walkom wrote:
>
> It's being looked at, but I don't know much beyond that at the moment 
> sorry.
>
> On 21 November 2014 20:02, > wrote:
>
>> Is there any of the elasticsearch team members that can hint to whether 
>> or not this is something that will be fixed in 1.4.1? Then we'll simply 
>> wait for it instead of doing different hacks to upgrade.
>>
>> On Monday, November 17, 2014 12:35:03 PM UTC+1, Matthew Barrington wrote:
>>>
>>> I stand corrected, this did not work on our main cluster.
>>>
>>> On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:

 We are running a 1.3.4 cluster using the AWS plugin and I noticed the 
 same error when I tried to upgrade a single node.

 Since I was trying this on my test cluster first I decided to see what 
 would happen if I upgraded a 2nd node. Would it split into 2 clusters, 
 have 
 the same issue, etc.

 What I discovered was that when 2 nodes were upgraded to 1.4 they 
 joined the cluster correctly and everything looks to be working.

 SO the problem seems to be for the initial node to join, but when you 
 try with two everything works out.

 On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>
> On Fri, Nov 14, 2014 at 3:41 AM,   wrote: 
> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
> cluster 
> > with cloud-aws plugin version 2.4.0. Is there a workaround to use 
> during 
> > upgrade, since I assume it's not a problem when they're all upgraded 
> to 
> > 1.4.0. 
>
> I ended up starting a new cluster (ignoring all the warnings logged on 
> startup), and restoring from a snapshot. Once all the 1.3.4 nodes were 
> gone, no issues. 
>
> -- 
> Eric Jain 
> Got data? Get answers at zenobase.com. 
>
  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/312dcdc1-d826-4cb9-b480-620232634ea7%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8decfa8a-7583-41ad-ba0f-f7982e49b73d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Mark Walkom]

It's being looked at, but I don't know much beyond that at the moment sorry.

On 21 November 2014 20:02,  wrote:

> Is there any of the elasticsearch team members that can hint to whether or
> not this is something that will be fixed in 1.4.1? Then we'll simply wait
> for it instead of doing different hacks to upgrade.
>
> On Monday, November 17, 2014 12:35:03 PM UTC+1, Matthew Barrington wrote:
>>
>> I stand corrected, this did not work on our main cluster.
>>
>> On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:
>>>
>>> We are running a 1.3.4 cluster using the AWS plugin and I noticed the
>>> same error when I tried to upgrade a single node.
>>>
>>> Since I was trying this on my test cluster first I decided to see what
>>> would happen if I upgraded a 2nd node. Would it split into 2 clusters, have
>>> the same issue, etc.
>>>
>>> What I discovered was that when 2 nodes were upgraded to 1.4 they joined
>>> the cluster correctly and everything looks to be working.
>>>
>>> SO the problem seems to be for the initial node to join, but when you
>>> try with two everything works out.
>>>
>>> On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:

 On Fri, Nov 14, 2014 at 3:41 AM,   wrote:
 > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4
 cluster
 > with cloud-aws plugin version 2.4.0. Is there a workaround to use
 during
 > upgrade, since I assume it's not a problem when they're all upgraded
 to
 > 1.4.0.

 I ended up starting a new cluster (ignoring all the warnings logged on
 startup), and restoring from a snapshot. Once all the 1.3.4 nodes were
 gone, no issues.

 --
 Eric Jain
 Got data? Get answers at zenobase.com.

>>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/312dcdc1-d826-4cb9-b480-620232634ea7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAF3ZnZn-ryDDoQps-smzUPkJd5ru9EHfKuAGRReU2-J-C35kvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[madsmartin]

Is there any of the elasticsearch team members that can hint to whether or 
not this is something that will be fixed in 1.4.1? Then we'll simply wait 
for it instead of doing different hacks to upgrade.

On Monday, November 17, 2014 12:35:03 PM UTC+1, Matthew Barrington wrote:
>
> I stand corrected, this did not work on our main cluster.
>
> On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:
>>
>> We are running a 1.3.4 cluster using the AWS plugin and I noticed the 
>> same error when I tried to upgrade a single node.
>>
>> Since I was trying this on my test cluster first I decided to see what 
>> would happen if I upgraded a 2nd node. Would it split into 2 clusters, have 
>> the same issue, etc.
>>
>> What I discovered was that when 2 nodes were upgraded to 1.4 they joined 
>> the cluster correctly and everything looks to be working.
>>
>> SO the problem seems to be for the initial node to join, but when you try 
>> with two everything works out.
>>
>> On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>>>
>>> On Fri, Nov 14, 2014 at 3:41 AM,   wrote: 
>>> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
>>> cluster 
>>> > with cloud-aws plugin version 2.4.0. Is there a workaround to use 
>>> during 
>>> > upgrade, since I assume it's not a problem when they're all upgraded 
>>> to 
>>> > 1.4.0. 
>>>
>>> I ended up starting a new cluster (ignoring all the warnings logged on 
>>> startup), and restoring from a snapshot. Once all the 1.3.4 nodes were 
>>> gone, no issues. 
>>>
>>> -- 
>>> Eric Jain 
>>> Got data? Get answers at zenobase.com. 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/312dcdc1-d826-4cb9-b480-620232634ea7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: 1.4.0 data node can't join existing 1.3.4 cluster

[Christian Hedegaard]

FYI, I have found a solution that works (at least for me).

I’ve got a small cluster for testing, only 4 v1.3.5 nodes. What I’ve done is 
bring up 4X new v1.4.0 nodes as data-only machines. In the yaml I added a line 
to point the nodes via unicast explicitly to the current master:
discovery.zen.ping.unicast.hosts: ["10.210.9.224:9300"]

When I restarted elasticsearch with that setting, with cloud-aws installed and 
configured on version 2.4.0, the new nodes found the cluster and properly 
joined it.

I will now start nuking the old v1.3.5 nodes to migrate the data off of them. 
Before the final 1.3.5 node is nuked, I will change the config on one of the 
v1.4.0 nodes to allow it as master and restart it.

I’m not sure if the master stuff is needed or not, but I was very afraid of a 
split-brain problem. I have another 4-node testing cluster that I will be able 
to try this upgrade again with in a more controlled manner.

I’m NOT looking forward to upgrading our current production cluster this way 
(15 data-only nodes, 3 master-only nodes).

So it would appear that the problem is somewhere in the unicast discovery code. 
The question is who’s to blame? Elasticsearch or the cloud-aws plugin?



From: Boaz Leskes [mailto:b.les...@gmail.com]
Sent: Wednesday, November 19, 2014 2:27 PM
To: elasticsearch@googlegroups.com
Cc: Christian Hedegaard
Subject: Re: 1.4.0 data node can't join existing 1.3.4 cluster

Hi Christian,

I'm not sure what thread you refer to exactly, but this shouldn't happen. Can 
you describe the problem you have some more? Anything in the nodes? (both the 
1.4 node and the master)

Cheers,
Boaz

On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard wrote:
I found this thread while trying to research the same issue and it looks like 
there is currently no resolution. We like to keep up on our elasticsearch 
upgrades as often as possible and do rolling upgrades to keep our clusters up. 
When testing I’m having the same issue, I cannot add a 1.4.0 box to the 
existing 1.3.4 cluster.

Is there a fix for this anticipated?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5CF8216AA982AF47A8E6DEACA629D22B4EBF409B%40s-us-ex-6.US.R5S.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Daniel Schonfeld]

We use GCE and the GCE plugin, on initial load it seemed like what you are 
describing.  Only when I examined the logs carefully i noticed that its 
more strict in reading the yaml and in our case the zone used to be

cloud:
  gce:
  zone: ["us-central1-a","us-central1-f"]

And with 1.4 it should now be:

cloud:
  gce:
zone: us-central1-a,us-central1-f

I haven't looked into the AWS plugin, but perhaps it the same kind of 
strict yaml configuration read.

Good luck!

On Wednesday, November 19, 2014 5:43:49 PM UTC-5, Christian Hedegaard wrote:
>
>  From the archives: 
> https://groups.google.com/forum/#!searchin/elasticsearch/1.4$20data$20node/elasticsearch/8pUwFld88tI/sBH7bB7rYzsJ
>
>  
>
> Same subject as mine. Started on the 13th.
>
>  
>
> Anyways, I’m having the exact same issue. I’ve got a cluster on 1.3.4 
> (well, now I’ve upgraded it to 1.3.5). When I provision a new node with 1.4 
> and the cloud-aws plugin with the proper version (2.4.0), the new node will 
> not come up and join the cluster and so a rolling upgrade does not appear 
> possible.
>
>  
>
>  
>
> *From:* Boaz Leskes [mailto:b.le...@gmail.com ] 
> *Sent:* Wednesday, November 19, 2014 2:27 PM
> *To:* elasti...@googlegroups.com 
> *Cc:* Christian Hedegaard
> *Subject:* Re: 1.4.0 data node can't join existing 1.3.4 cluster
>
>  
>  
> Hi Christian, 
>  
>  
>  
> I'm not sure what thread you refer to exactly, but this shouldn't happen. 
> Can you describe the problem you have some more? Anything in the nodes? 
> (both the 1.4 node and the master)
>  
>  
>  
> Cheers,
>  
> Boaz
>  
>
> On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard 
> wrote:
>  
> I found this thread while trying to research the same issue and it looks 
> like there is currently no resolution. We like to keep up on our 
> elasticsearch upgrades as often as possible and do rolling upgrades to keep 
> our clusters up. When testing I’m having the same issue, I cannot add a 
> 1.4.0 box to the existing 1.3.4 cluster.
>
>  
>
> Is there a fix for this anticipated?
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/294e7c85-10a0-4975-a3ef-12fd106cd69c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: 1.4.0 data node can't join existing 1.3.4 cluster

[Christian Hedegaard]

>From the archives: 
>https://groups.google.com/forum/#!searchin/elasticsearch/1.4$20data$20node/elasticsearch/8pUwFld88tI/sBH7bB7rYzsJ

Same subject as mine. Started on the 13th.

Anyways, I’m having the exact same issue. I’ve got a cluster on 1.3.4 (well, 
now I’ve upgraded it to 1.3.5). When I provision a new node with 1.4 and the 
cloud-aws plugin with the proper version (2.4.0), the new node will not come up 
and join the cluster and so a rolling upgrade does not appear possible.


From: Boaz Leskes [mailto:b.les...@gmail.com]
Sent: Wednesday, November 19, 2014 2:27 PM
To: elasticsearch@googlegroups.com
Cc: Christian Hedegaard
Subject: Re: 1.4.0 data node can't join existing 1.3.4 cluster

Hi Christian,

I'm not sure what thread you refer to exactly, but this shouldn't happen. Can 
you describe the problem you have some more? Anything in the nodes? (both the 
1.4 node and the master)

Cheers,
Boaz

On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard wrote:
I found this thread while trying to research the same issue and it looks like 
there is currently no resolution. We like to keep up on our elasticsearch 
upgrades as often as possible and do rolling upgrades to keep our clusters up. 
When testing I’m having the same issue, I cannot add a 1.4.0 box to the 
existing 1.3.4 cluster.

Is there a fix for this anticipated?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5CF8216AA982AF47A8E6DEACA629D22B4EBF0DA6%40s-us-ex-6.US.R5S.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Boaz Leskes]

Hi Christian, 

I'm not sure what thread you refer to exactly, but this shouldn't happen. 
Can you describe the problem you have some more? Anything in the nodes? 
(both the 1.4 node and the master)

Cheers,
Boaz

On Wednesday, November 19, 2014 2:39:57 AM UTC+1, Christian Hedegaard wrote:
>
>  I found this thread while trying to research the same issue and it looks 
> like there is currently no resolution. We like to keep up on our 
> elasticsearch upgrades as often as possible and do rolling upgrades to keep 
> our clusters up. When testing I’m having the same issue, I cannot add a 
> 1.4.0 box to the existing 1.3.4 cluster.
>
>  
>
> Is there a fix for this anticipated?
>  

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/16f33a69-fb07-4db1-9c05-b9031c867a63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1.4.0 data node can't join existing 1.3.4 cluster

[Christian Hedegaard]

I found this thread while trying to research the same issue and it looks like 
there is currently no resolution. We like to keep up on our elasticsearch 
upgrades as often as possible and do rolling upgrades to keep our clusters up. 
When testing I'm having the same issue, I cannot add a 1.4.0 box to the 
existing 1.3.4 cluster.

Is there a fix for this anticipated?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5CF8216AA982AF47A8E6DEACA629D22B4EBEEF75%40s-us-ex-6.US.R5S.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Matthew Barrington]

I stand corrected, this did not work on our main cluster.

On Monday, 17 November 2014 11:13:22 UTC, Matthew Barrington wrote:
>
> We are running a 1.3.4 cluster using the AWS plugin and I noticed the same 
> error when I tried to upgrade a single node.
>
> Since I was trying this on my test cluster first I decided to see what 
> would happen if I upgraded a 2nd node. Would it split into 2 clusters, have 
> the same issue, etc.
>
> What I discovered was that when 2 nodes were upgraded to 1.4 they joined 
> the cluster correctly and everything looks to be working.
>
> SO the problem seems to be for the initial node to join, but when you try 
> with two everything works out.
>
> On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>>
>> On Fri, Nov 14, 2014 at 3:41 AM,   wrote: 
>> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
>> cluster 
>> > with cloud-aws plugin version 2.4.0. Is there a workaround to use 
>> during 
>> > upgrade, since I assume it's not a problem when they're all upgraded to 
>> > 1.4.0. 
>>
>> I ended up starting a new cluster (ignoring all the warnings logged on 
>> startup), and restoring from a snapshot. Once all the 1.3.4 nodes were 
>> gone, no issues. 
>>
>> -- 
>> Eric Jain 
>> Got data? Get answers at zenobase.com. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/fc79c5e7-ddd6-4f52-9641-1bd01df3b866%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Matthew Barrington]

We are running a 1.3.4 cluster using the AWS plugin and I noticed the same 
error when I tried to upgrade a single node.

Since I was trying this on my test cluster first I decided to see what 
would happen if I upgraded a 2nd node. Would it split into 2 clusters, have 
the same issue, etc.

What I discovered was that when 2 nodes were upgraded to 1.4 they joined 
the cluster correctly and everything looks to be working.

SO the problem seems to be for the initial node to join, but when you try 
with two everything works out.

On Friday, 14 November 2014 18:05:01 UTC, Eric Jain wrote:
>
> On Fri, Nov 14, 2014 at 3:41 AM,  > 
> wrote: 
> > I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
> cluster 
> > with cloud-aws plugin version 2.4.0. Is there a workaround to use during 
> > upgrade, since I assume it's not a problem when they're all upgraded to 
> > 1.4.0. 
>
> I ended up starting a new cluster (ignoring all the warnings logged on 
> startup), and restoring from a snapshot. Once all the 1.3.4 nodes were 
> gone, no issues. 
>
> -- 
> Eric Jain 
> Got data? Get answers at zenobase.com. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f618a79d-0a4f-4f97-8b89-ab6ccb9d1cbe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Eric Jain]

On Fri, Nov 14, 2014 at 3:41 AM,   wrote:
> I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 cluster
> with cloud-aws plugin version 2.4.0. Is there a workaround to use during
> upgrade, since I assume it's not a problem when they're all upgraded to
> 1.4.0.

I ended up starting a new cluster (ignoring all the warnings logged on
startup), and restoring from a snapshot. Once all the 1.3.4 nodes were
gone, no issues.

-- 
Eric Jain
Got data? Get answers at zenobase.com.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHte5%2BJ%2BfnBnus%3DOX6itdwcaB9%2Bh_KDMwDYRBWU-4fWL0CohJA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[madsmartin]

1.4.0 trying to join a 1.3.5 cluster with cloud-aws also fails.

On Friday, November 14, 2014 12:41:08 PM UTC+1, madsm...@colourbox.com 
wrote:
>
> I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 
> cluster with cloud-aws plugin version 2.4.0. Is there a workaround to use 
> during upgrade, since I assume it's not a problem when they're all upgraded 
> to 1.4.0.
>
>
> On Friday, November 14, 2014 11:33:45 AM UTC+1, Jörg Prante wrote:
>>
>> I think this is only related to unicast. But, nevertheless, it *should* 
>> work... not sure if this is a bug or a feature
>>
>> Jörg
>>
>> On Fri, Nov 14, 2014 at 12:58 AM, Eric Jain  wrote:
>>
>>> On Thu, Nov 13, 2014 at 10:05 AM, joerg...@gmail.com
>>>  wrote:
>>> > Do not mix 1.3 with 1.4 nodes, it does not work.
>>>
>>> If that is so, that seems like something the release notes should 
>>> mention?
>>>
>>> --
>>> You received this message because you are subscribed to the Google 
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/CAHte5%2BKyW3nZ82uf1w1C_i3O9oW%3D%3DhG7f3WUC2-g40%3DwUn5Fgw%40mail.gmail.com
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/967ce7d9-3f53-4478-868b-85443a904620%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[madsmartin]

I'm also seing this problem when a 1.4.0 node tries joining a 1.3.4 cluster 
with cloud-aws plugin version 2.4.0. Is there a workaround to use during 
upgrade, since I assume it's not a problem when they're all upgraded to 
1.4.0.


On Friday, November 14, 2014 11:33:45 AM UTC+1, Jörg Prante wrote:
>
> I think this is only related to unicast. But, nevertheless, it *should* 
> work... not sure if this is a bug or a feature
>
> Jörg
>
> On Fri, Nov 14, 2014 at 12:58 AM, Eric Jain  > wrote:
>
>> On Thu, Nov 13, 2014 at 10:05 AM, joerg...@gmail.com 
>> > wrote:
>> > Do not mix 1.3 with 1.4 nodes, it does not work.
>>
>> If that is so, that seems like something the release notes should mention?
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/CAHte5%2BKyW3nZ82uf1w1C_i3O9oW%3D%3DhG7f3WUC2-g40%3DwUn5Fgw%40mail.gmail.com
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/23b1428c-1e55-498e-ac8f-499adc8528e0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[joergpra...@gmail.com]

I think this is only related to unicast. But, nevertheless, it *should*
work... not sure if this is a bug or a feature

Jörg

On Fri, Nov 14, 2014 at 12:58 AM, Eric Jain  wrote:

> On Thu, Nov 13, 2014 at 10:05 AM, joergpra...@gmail.com
>  wrote:
> > Do not mix 1.3 with 1.4 nodes, it does not work.
>
> If that is so, that seems like something the release notes should mention?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAHte5%2BKyW3nZ82uf1w1C_i3O9oW%3D%3DhG7f3WUC2-g40%3DwUn5Fgw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE7GvQiiz2G-PjNWXNKg5NWHE%2BUfwE0bg_mHekw6e5WZw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Eric Jain]

On Thu, Nov 13, 2014 at 10:05 AM, joergpra...@gmail.com
 wrote:
> Do not mix 1.3 with 1.4 nodes, it does not work.

If that is so, that seems like something the release notes should mention?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAHte5%2BKyW3nZ82uf1w1C_i3O9oW%3D%3DhG7f3WUC2-g40%3DwUn5Fgw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[joergpra...@gmail.com]

I discovered a peculiarity, well documented in the source

https://github.com/elasticsearch/elasticsearch/commit/e5de47d928582694c7729d199390086983779e6e

/**
 * when pinging the initial configured target hosts, we do not know
their version. We therefore use
 * the lowest possible version (i.e., 1.0.0) for serializing
information on the wire. As of 1.4, we needed to extend
 * the information sent in a ping, to prefer nodes which have
previously joined the cluster during master election.
 * This information is only needed if all the cluster is on version 1.4
or up. To bypass this issue we introduce
 * a second action name which is guaranteed to exist only on nodes from
version 1.4.0 and up. Using this action,
 * we can safely use 1.4.0 as a serialization format. If this fails
with a {@link ActionNotFoundTransportException}
 * we know we speak to a node with <1.4 version, and fall back to use
{@link #ACTION_NAME}.
 */
public static final String ACTION_NAME_GTE_1_4 =
"internal:discovery/zen/unicast_gte_1_4";

Not sure if everything works well...

Jörg


On Thu, Nov 13, 2014 at 10:17 PM, Ivan Brusic  wrote:

> Rolling upgrades should be supported:
>
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-upgrade.html#rolling-upgrades
>
> How else can you perform a rolling upgrade without having a mixed cluster?
>
> --
> Ivan
>
> On Thu, Nov 13, 2014 at 1:05 PM, joergpra...@gmail.com <
> joergpra...@gmail.com> wrote:
>
>> Do not mix 1.3 with 1.4 nodes, it does not work.
>>
>> Jörg
>>
>> On Thu, Nov 13, 2014 at 5:16 PM, Todd Kamin  wrote:
>>
>>> I'm seeing something very similar, also from 1.3.4 to 1.4.0, also using
>>> the elasticsearch-cloud-aws plugin 2.4.
>>>
>>> [2014-11-13 16:02:26,055][WARN ][discovery.zen.ping.unicast] [White
>>> Fang] failed to send ping to
>>> [[#cloud-i-03f79bcb-0][localhost][inet[/10.0.0.76:9300]]]
>>> org.elasticsearch.transport.RemoteTransportException: [Kraven the
>>> Hunter][inet[/10.0.0.76:9300]][internal:discovery/zen/unicast]
>>> Caused by: org.elasticsearch.transport.ActionNotFoundTransportException:
>>> No handler for action [internal:discovery/zen/unicast]
>>> at
>>> org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:210)
>>> at
>>> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:111)
>>>
>>>  --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/840f26f7-c401-4229-b94d-092ee2ef4974%40googlegroups.com
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGOOeMcuZqw7xADCuTc-MTQkuqQ0QtnBR%2Bg9TWBS6DOng%40mail.gmail.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDrfhTAc6FhPpU-ZcHtW4Qp_9qB9G6z%2BSXUR7fqtB52_A%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFN06R7-sYGcsaUtg_VagcjSHikbmPw%3DUSz-H29nvrmWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Ivan Brusic]

Rolling upgrades should be supported:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-upgrade.html#rolling-upgrades

How else can you perform a rolling upgrade without having a mixed cluster?

-- 
Ivan

On Thu, Nov 13, 2014 at 1:05 PM, joergpra...@gmail.com <
joergpra...@gmail.com> wrote:

> Do not mix 1.3 with 1.4 nodes, it does not work.
>
> Jörg
>
> On Thu, Nov 13, 2014 at 5:16 PM, Todd Kamin  wrote:
>
>> I'm seeing something very similar, also from 1.3.4 to 1.4.0, also using
>> the elasticsearch-cloud-aws plugin 2.4.
>>
>> [2014-11-13 16:02:26,055][WARN ][discovery.zen.ping.unicast] [White Fang]
>> failed to send ping to
>> [[#cloud-i-03f79bcb-0][localhost][inet[/10.0.0.76:9300]]]
>> org.elasticsearch.transport.RemoteTransportException: [Kraven the
>> Hunter][inet[/10.0.0.76:9300]][internal:discovery/zen/unicast]
>> Caused by: org.elasticsearch.transport.ActionNotFoundTransportException:
>> No handler for action [internal:discovery/zen/unicast]
>> at
>> org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:210)
>> at
>> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:111)
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/840f26f7-c401-4229-b94d-092ee2ef4974%40googlegroups.com
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGOOeMcuZqw7xADCuTc-MTQkuqQ0QtnBR%2Bg9TWBS6DOng%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDrfhTAc6FhPpU-ZcHtW4Qp_9qB9G6z%2BSXUR7fqtB52_A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[joergpra...@gmail.com]

Do not mix 1.3 with 1.4 nodes, it does not work.

Jörg

On Thu, Nov 13, 2014 at 5:16 PM, Todd Kamin  wrote:

> I'm seeing something very similar, also from 1.3.4 to 1.4.0, also using
> the elasticsearch-cloud-aws plugin 2.4.
>
> [2014-11-13 16:02:26,055][WARN ][discovery.zen.ping.unicast] [White Fang]
> failed to send ping to
> [[#cloud-i-03f79bcb-0][localhost][inet[/10.0.0.76:9300]]]
> org.elasticsearch.transport.RemoteTransportException: [Kraven the
> Hunter][inet[/10.0.0.76:9300]][internal:discovery/zen/unicast]
> Caused by: org.elasticsearch.transport.ActionNotFoundTransportException:
> No handler for action [internal:discovery/zen/unicast]
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:210)
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:111)
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/840f26f7-c401-4229-b94d-092ee2ef4974%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGOOeMcuZqw7xADCuTc-MTQkuqQ0QtnBR%2Bg9TWBS6DOng%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.4.0 data node can't join existing 1.3.4 cluster

[Todd Kamin]

I'm seeing something very similar, also from 1.3.4 to 1.4.0, also using the 
elasticsearch-cloud-aws plugin 2.4.

[2014-11-13 16:02:26,055][WARN ][discovery.zen.ping.unicast] [White Fang] 
failed to send ping to 
[[#cloud-i-03f79bcb-0][localhost][inet[/10.0.0.76:9300]]]
org.elasticsearch.transport.RemoteTransportException: [Kraven the 
Hunter][inet[/10.0.0.76:9300]][internal:discovery/zen/unicast]
Caused by: org.elasticsearch.transport.ActionNotFoundTransportException: No 
handler for action [internal:discovery/zen/unicast]
at 
org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:210)
at 
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:111)

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/840f26f7-c401-4229-b94d-092ee2ef4974%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1.4.0 data node can't join existing 1.3.4 cluster

[Eric Jain]

(using elasticsearch-cloud-aws 2.4)

This should work, right? Or do I need to upgrade the cluster to 1.3.5 first?

The connection fails after a few errors like:

2014-11-13 07:18:22,498 [WARN] org.elasticsearch.discovery.zen.ping.unicast 
- [Porcupine] failed to send ping to 
[[#cloud-i-b743e456-0][530-1d][inet[/10.186.145.210:9300]]]
org.elasticsearch.transport.RemoteTransportException: 
[Nomad][inet[/10.186.145.210:9300]][internal:discovery/zen/unicast]
Caused by: org.elasticsearch.transport.ActionNotFoundTransportException: No 
handler for action [internal:discovery/zen/unicast]
at 
org.elasticsearch.transport.netty.MessageChannelHandler.handleRequest(MessageChannelHandler.java:210)
 
~[org.elasticsearch.elasticsearch-1.4.0.jar:na]
at 
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:111)
 
~[org.elasticsearch.elasticsearch-1.4.0.jar:na]
at 
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
 
~[org.elasticsearch.elasticsearch-1.4.0.jar:na]

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/efd3232c-b22f-4f2b-94c4-c942b75e81bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.