Hello,
I am attempting to upgrade a 10 node cluster from 1.3.2 to 1.4.2 - I
upgraded the first node,removed and reinstalled latest versions of plugins
- two non-site plugins are river-twitter (version 2.4.1) and jdbc (version
1.4.0.8 and also tried 1.4.0.7) - and when starting the node I see the
errors below in logs on upgraded node and node does not join cluster. If i
downgrade to 1.3.2, uninstall plugins and reinstall jdbc river plugin
version 1.3.0.4 the node properly joins the cluster.
Errors from logs:
[2015-01-06 21:29:39,970][INFO ][node ] [bi-es1]
version[1.4.2], pid[5910], build[927caff/2014-12-16T14:11:12Z]
[2015-01-06 21:29:39,971][INFO ][node ] [bi-es1]
initializing ...
[2015-01-06 21:29:40,009][INFO ][plugins ] [bi-es1] loaded
[river-twitter, jdbc-1.4.0.7-a875ced], sites [head, kopf, bigdesk,
paramedic, HQ, whatson]
[2015-01-06 21:29:45,587][INFO ][node ] [bi-es1]
initialized
[2015-01-06 21:29:45,588][INFO ][node ] [bi-es1]
starting ...
[2015-01-06 21:29:45,960][INFO ][transport ] [bi-es1]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/192.168.83.231:9300]}
[2015-01-06 21:29:45,982][INFO ][discovery ] [bi-es1]
bi-cluster1/VyTzSKWnQAS9ZCk971fJEg
[2015-01-06 21:29:51,906][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28805963] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:29:51,915][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:29:57,028][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28807127] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:29:57,036][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:30:02,245][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28808254] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:30:02,252][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:30:07,576][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28809377] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:30:07,583][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:30:12,689][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28810050] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:30:12,696][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:30:15,989][WARN ][discovery ] [bi-es1] waited
for 30s and no initial state was set by the discovery
[2015-01-06 21:30:16,002][INFO ][http ] [bi-es1]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/192.168.83.231:9200]}
[2015-01-06 21:30:16,003][INFO ][node ] [bi-es1] started
[2015-01-06 21:30:18,277][WARN ][transport.netty ] [bi-es1]
Message not fully read (request) for [28811164] and action
[discovery/zen/join/validate], resetting
[2015-01-06 21:30:18,292][INFO ][discovery.zen ] [bi-es1] failed
to send join request to master
[[bi-es4][-cczOZfKTw-duMdUO_YSAw][bi-es4][inet[/192.168.83.234:9300]]{master=true}],
reason
[RemoteTransportException[[bi-es4][inet[/192.168.83.234:9300]][discovery/zen/join]];
nested:
RemoteTransportException[[bi-es1][inet[/192.168.83.231:9300]][discovery/zen/join/validate]];
nested: ElasticsearchIllegalArgumentException[No custom index metadata
factory registered for type [rivers]]; ]
[2015-01-06 21:30:19,406][DEBUG][action.admin.cluster.state] [bi-es1] no
known master node, scheduling a retry
I have tried removing all plugins from the upgraded node and that results
in same errors. I have tried closing the _river index and removing all
defined rivers (wasn't hopeful that would work but gave it a shot).
I am not sure what else to try - thoughts? ideas?
I am safely downgraded back to 1.3.2 and node rejoined and things seem good
except still on 1.3.2. Would prefer not to do a full cluster shutdown to
upgrade all nodes and jdbc plugin but that seems maybe it would solve this?
I have had issues in past with upgrades/jdbc river plugin (really love the
plugin though) so my brain is focusing on that and might not be the issue?
I used same process on a test cluster that I just realized did not have the
jdbc plugin installed on it and that went smooth.
Any help would be greatly appreciated!
Thanks
Ben
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ef4334fc-b6c9-4802-ab8a-934989901efd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
