Re: Shards UNASSIGNED even tho they exist on disk
Sounds like you don't have enough space on disk(s)
Happened to me when upgrading to 1.4.
On Monday, November 10, 2014 2:45:09 PM UTC+1, Johan Öhr wrote:
Hi,
I have a problem with a few index, some of the shards (both replica and
primary) are UNASSIGNED, my cluster stays yellow.
This is what the master says about that:
[2014-11-10 06:53:01,223][WARN ][cluster.action.shard ] [node-master]
[index][9] received shard failed for [index][9],
node[9g2_kOrDSt-57UVI1bLfFg], [P], s[STARTED], indexUUID
[20P5SMNFTZyrUEVyUPCsbQ], reason [master
[node-master][07ZcjsurR3iIVsH6iSX0jw][data-node][inet[/xx.xx.xx.xx:9300]]{data=false,
master=true} marked shard as started, but shard has not been created, mark
shard as failed]
http://host:9200/index/_stats_shards:{failed:0,successful:13,
total:20
This happend when i dropped a node, and let it replicate itself together,
replication factor is 1 (two shards identical)
I did it on two nodes, worked perfectly, then on the third node, i have 92
SHARDS Unassigned
The only different between the first two nodes and the third is that it
ran with these settings:
cluster.routing.allocation.disk.threshold_enabled: true,
cluster.routing.allocation.disk.watermark.low: 0.85,
cluster.routing.allocation.disk.watermark.high: 0.90,
cluster.info.update.interval: 60s,
indices.recovery.concurrent_streams: 10,
cluster.routing.allocation.node_concurrent_recoveries: 40,
Any idea if this can be fixed?
Ive tried to clean up the masters and restarted them, nothing
Ive tried to delete _state on data-node on these index, nothing
Thanks for help :)
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c1c3bd8-1e94-471d-ab06-fc643afe836e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Shards UNASSIGNED even tho they exist on disk
Hi,
I have a problem with a few index, some of the shards (both replica and
primary) are UNASSIGNED, my cluster stays yellow.
This is what the master says about that:
[2014-11-10 06:53:01,223][WARN ][cluster.action.shard ] [node-master]
[index][9] received shard failed for [index][9],
node[9g2_kOrDSt-57UVI1bLfFg], [P], s[STARTED], indexUUID
[20P5SMNFTZyrUEVyUPCsbQ], reason [master
[node-master][07ZcjsurR3iIVsH6iSX0jw][data-node][inet[/xx.xx.xx.xx:9300]]{data=false,
master=true} marked shard as started, but shard has not been created, mark
shard as failed]
http://host:9200/index/_stats_shards:{failed:0,successful:13,total:
20
This happend when i dropped a node, and let it replicate itself together,
replication factor is 1 (two shards identical)
I did it on two nodes, worked perfectly, then on the third node, i have 92
SHARDS Unassigned
The only different between the first two nodes and the third is that it ran
with these settings:
cluster.routing.allocation.disk.threshold_enabled: true,
cluster.routing.allocation.disk.watermark.low: 0.85,
cluster.routing.allocation.disk.watermark.high: 0.90,
cluster.info.update.interval: 60s,
indices.recovery.concurrent_streams: 10,
cluster.routing.allocation.node_concurrent_recoveries: 40,
Any idea if this can be fixed?
Ive tried to clean up the masters and restarted them, nothing
Ive tried to delete _state on data-node on these index, nothing
Thanks for help :)
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1706f0bc-5aef-42dc-bcbe-8f62efd25faf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Shards UNASSIGNED even tho they exist on disk
Another thing i just noticed..
The first problematic index with the problem, is from 2014-10-08,
We upgraded to 1.3.2 from 1.01 2014-10-07, indexes is created nighthly.
Den måndagen den 10:e november 2014 kl. 14:45:09 UTC+1 skrev Johan Öhr:
Hi,
I have a problem with a few index, some of the shards (both replica and
primary) are UNASSIGNED, my cluster stays yellow.
This is what the master says about that:
[2014-11-10 06:53:01,223][WARN ][cluster.action.shard ] [node-master]
[index][9] received shard failed for [index][9],
node[9g2_kOrDSt-57UVI1bLfFg], [P], s[STARTED], indexUUID
[20P5SMNFTZyrUEVyUPCsbQ], reason [master
[node-master][07ZcjsurR3iIVsH6iSX0jw][data-node][inet[/xx.xx.xx.xx:9300]]{data=false,
master=true} marked shard as started, but shard has not been created, mark
shard as failed]
http://host:9200/index/_stats_shards:{failed:0,successful:13,
total:20
This happend when i dropped a node, and let it replicate itself together,
replication factor is 1 (two shards identical)
I did it on two nodes, worked perfectly, then on the third node, i have 92
SHARDS Unassigned
The only different between the first two nodes and the third is that it
ran with these settings:
cluster.routing.allocation.disk.threshold_enabled: true,
cluster.routing.allocation.disk.watermark.low: 0.85,
cluster.routing.allocation.disk.watermark.high: 0.90,
cluster.info.update.interval: 60s,
indices.recovery.concurrent_streams: 10,
cluster.routing.allocation.node_concurrent_recoveries: 40,
Any idea if this can be fixed?
Ive tried to clean up the masters and restarted them, nothing
Ive tried to delete _state on data-node on these index, nothing
Thanks for help :)
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/03686f7e-df0b-4fcd-89fd-ebc66a0ebc05%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
