Re: elasticsearch deploy on ec2
Can you describe your deployment process? the cluster can't be _always_ red
- it should be green when you first deploy
Other than that, check the obvious - that AWS securty groups are properly
defined for those machines (all of them under the same named security group)
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer Consultant
Lucene.NET committer and PMC member
On Mon, Feb 16, 2015 at 8:11 PM, Eliran Shlomo eli...@whipclip.com wrote:
Hi,
I'm trying to deploy new elasticsearch environment on aws and the cluster
is always at red and i get the following message
{
error: ClusterBlockException[blocked by:
[SERVICE_UNAVAILABLE/1/state not recovered / initialized];],
status: 503
}
I'm trying to deploy 3 master nodes, 3 data nodes, and 2 client nodes.
when i check cluster health:
{
cluster_name: stress_new,
status: red,
timed_out: false,
number_of_nodes: 8,
number_of_data_nodes: 3,
active_primary_shards: 0,
active_shards: 0,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0
}
please adivce
this is the configuration i used with changes by node role ( node.master
node.data ):
cluster.name: stress_new
plugin.mandatory: cloud-aws
cloud.aws.access_key:
cloud.aws.secret_key: *
cloud.aws.region: us-west-2
discovery.type: ec2
discovery.ec2.groups: stress_new_elasticsearch
discovery.ec2.host_type: private_ip
discovery.ec2.ping_timeout: 30s
discovery.ec2.tag.elasticsearch: stress_new
node.name: 172.**
node.master: false
node.data: false
index.number_of_shards: 1
index.number_of_replicas: 0
path.data: /mnt/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.mlockall: true
http.enabled: true
gateway.recover_after_nodes: 8
gateway.expected_nodes: 8
discovery.zen.minimum_master_nodes: 3
discovery.zen.ping.timeout: 10s
discovery.zen.ping.multicast.enabled: false
index.search.slowlog.threshold.query.warn: 500ms
index.search.slowlog.threshold.query.info: 200ms
index.search.slowlog.threshold.query.debug: 100ms
index.search.slowlog.threshold.query.trace: 50ms
index.search.slowlog.threshold.fetch.warn: 500ms
index.search.slowlog.threshold.fetch.info: 200ms
index.search.slowlog.threshold.fetch.debug: 100ms
index.search.slowlog.threshold.fetch.trace: 50ms
index.indexing.slowlog.threshold.index.warn: 500ms
index.indexing.slowlog.threshold.index.info: 200ms
index.indexing.slowlog.threshold.index.debug: 1000ms
index.indexing.slowlog.threshold.index.trace: 50ms
script.disable_dynamic: false
script.native.socialScoreCalc.type: *.SocialScriptFactory
script.default_lang: native
action.disable_delete_all_indices: true
action.auto_create_index: .marvel-*
indices.fielddata.cache.size: 30%
indices.fielddata.cache.expire: 15s
marvel.agent.enabled: false
allow_leading_wildcard: false
script.groovy.sandbox.enabled: false
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZstAUwTELHxOx%3DqWBuOjByEn3mhgdd93-kG3M%3DxQMSS7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
elasticsearch deploy on ec2
Hi,
I'm trying to deploy new elasticsearch environment on aws and the cluster
is always at red and i get the following message
{
error: ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state
not recovered / initialized];],
status: 503
}
I'm trying to deploy 3 master nodes, 3 data nodes, and 2 client nodes.
when i check cluster health:
{
cluster_name: stress_new,
status: red,
timed_out: false,
number_of_nodes: 8,
number_of_data_nodes: 3,
active_primary_shards: 0,
active_shards: 0,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0
}
please adivce
this is the configuration i used with changes by node role ( node.master
node.data ):
cluster.name: stress_new
plugin.mandatory: cloud-aws
cloud.aws.access_key:
cloud.aws.secret_key: *
cloud.aws.region: us-west-2
discovery.type: ec2
discovery.ec2.groups: stress_new_elasticsearch
discovery.ec2.host_type: private_ip
discovery.ec2.ping_timeout: 30s
discovery.ec2.tag.elasticsearch: stress_new
node.name: 172.**
node.master: false
node.data: false
index.number_of_shards: 1
index.number_of_replicas: 0
path.data: /mnt/elasticsearch
path.logs: /var/log/elasticsearch
bootstrap.mlockall: true
http.enabled: true
gateway.recover_after_nodes: 8
gateway.expected_nodes: 8
discovery.zen.minimum_master_nodes: 3
discovery.zen.ping.timeout: 10s
discovery.zen.ping.multicast.enabled: false
index.search.slowlog.threshold.query.warn: 500ms
index.search.slowlog.threshold.query.info: 200ms
index.search.slowlog.threshold.query.debug: 100ms
index.search.slowlog.threshold.query.trace: 50ms
index.search.slowlog.threshold.fetch.warn: 500ms
index.search.slowlog.threshold.fetch.info: 200ms
index.search.slowlog.threshold.fetch.debug: 100ms
index.search.slowlog.threshold.fetch.trace: 50ms
index.indexing.slowlog.threshold.index.warn: 500ms
index.indexing.slowlog.threshold.index.info: 200ms
index.indexing.slowlog.threshold.index.debug: 1000ms
index.indexing.slowlog.threshold.index.trace: 50ms
script.disable_dynamic: false
script.native.socialScoreCalc.type: *.SocialScriptFactory
script.default_lang: native
action.disable_delete_all_indices: true
action.auto_create_index: .marvel-*
indices.fielddata.cache.size: 30%
indices.fielddata.cache.expire: 15s
marvel.agent.enabled: false
allow_leading_wildcard: false
script.groovy.sandbox.enabled: false
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:22 PM, Eliran Shlomo eli...@whipclip.com wrote: Hi, Since the first moment the cluster in in red The servers are under the same security group and inside the security group i allow any/any between the servers. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ed1d0f69-93cc-4467-a0ad-bffe4b5175df%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtJEVnyXCDkEX7Hj%3DR%3Dvyy%3DUsUiiDSXC0EVhBMU2PaYXQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Master eligible nodes and Data nodes need to have this setting -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com wrote: Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtzjXeHrzHiRAitfOHASpXrvzXza8JPVObtjMwCHqMrVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Remove the number of nodes setting, if that doesn't help start looking at
the logs. I seen clusters on AWS that took some time to discover and
stabilize, it may also be that.
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer Consultant
Lucene.NET committer and PMC member
On Mon, Feb 16, 2015 at 8:56 PM, Eliran Shlomo eli...@whipclip.com wrote:
Hi, made the changes.
no change in the cluster status and the same response from the servers
{
cluster_name: stress_new,
status: red,
timed_out: false,
number_of_nodes: 8,
number_of_data_nodes: 3,
active_primary_shards: 0,
active_shards: 0,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0
}
get _status
{
error: ClusterBlockException[blocked by:
[SERVICE_UNAVAILABLE/1/state not recovered / initialized];],
status: 503
}
On Monday, February 16, 2015 at 8:53:11 PM UTC+2, Itamar Syn-Hershko wrote:
Master eligible nodes and Data nodes need to have this setting
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer Consultant
Lucene.NET committer and PMC member
On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com
wrote:
Wait a second, you should use gateway.expected_data_nodes: 3 and
gateway.expected_master_nodes: 3 instead of what you have there now. Also
min master nodes should be 2 in your case.
Those settings should be in the configuration of all nodes or only in
the external gateway?(client)
--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%
40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zsa0DgkCJuuLrhyrssM%2BEFMQ4jcZwECWmq%3Dq0GFyqHVEg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Hi, Since the first moment the cluster in in red The servers are under the same security group and inside the security group i allow any/any between the servers. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ed1d0f69-93cc-4467-a0ad-bffe4b5175df%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Tnx for the help Itamar.
solved by changing gateway.recover_after_nodes value from 8 to 6
i guess client node shouldn't be count for recover
On Monday, February 16, 2015 at 8:59:48 PM UTC+2, Itamar Syn-Hershko wrote:
Remove the number of nodes setting, if that doesn't help start looking at
the logs. I seen clusters on AWS that took some time to discover and
stabilize, it may also be that.
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer Consultant
Lucene.NET committer and PMC member
On Mon, Feb 16, 2015 at 8:56 PM, Eliran Shlomo eli...@whipclip.com
javascript: wrote:
Hi, made the changes.
no change in the cluster status and the same response from the servers
{
cluster_name: stress_new,
status: red,
timed_out: false,
number_of_nodes: 8,
number_of_data_nodes: 3,
active_primary_shards: 0,
active_shards: 0,
relocating_shards: 0,
initializing_shards: 0,
unassigned_shards: 0
}
get _status
{
error: ClusterBlockException[blocked by:
[SERVICE_UNAVAILABLE/1/state not recovered / initialized];],
status: 503
}
On Monday, February 16, 2015 at 8:53:11 PM UTC+2, Itamar Syn-Hershko
wrote:
Master eligible nodes and Data nodes need to have this setting
--
Itamar Syn-Hershko
http://code972.com | @synhershko https://twitter.com/synhershko
Freelance Developer Consultant
Lucene.NET committer and PMC member
On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com
wrote:
Wait a second, you should use gateway.expected_data_nodes: 3 and
gateway.expected_master_nodes: 3 instead of what you have there now. Also
min master nodes should be 2 in your case.
Those settings should be in the configuration of all nodes or only in
the external gateway?(client)
--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%
40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9473e8bf-def6-4f04-89f3-b957394b4cf4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
