Re: elasticsearch deploy on ec2
Can you describe your deployment process? the cluster can't be _always_ red - it should be green when you first deploy Other than that, check the obvious - that AWS securty groups are properly defined for those machines (all of them under the same named security group) -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:11 PM, Eliran Shlomo eli...@whipclip.com wrote: Hi, I'm trying to deploy new elasticsearch environment on aws and the cluster is always at red and i get the following message { error: ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];], status: 503 } I'm trying to deploy 3 master nodes, 3 data nodes, and 2 client nodes. when i check cluster health: { cluster_name: stress_new, status: red, timed_out: false, number_of_nodes: 8, number_of_data_nodes: 3, active_primary_shards: 0, active_shards: 0, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0 } please adivce this is the configuration i used with changes by node role ( node.master node.data ): cluster.name: stress_new plugin.mandatory: cloud-aws cloud.aws.access_key: cloud.aws.secret_key: * cloud.aws.region: us-west-2 discovery.type: ec2 discovery.ec2.groups: stress_new_elasticsearch discovery.ec2.host_type: private_ip discovery.ec2.ping_timeout: 30s discovery.ec2.tag.elasticsearch: stress_new node.name: 172.** node.master: false node.data: false index.number_of_shards: 1 index.number_of_replicas: 0 path.data: /mnt/elasticsearch path.logs: /var/log/elasticsearch bootstrap.mlockall: true http.enabled: true gateway.recover_after_nodes: 8 gateway.expected_nodes: 8 discovery.zen.minimum_master_nodes: 3 discovery.zen.ping.timeout: 10s discovery.zen.ping.multicast.enabled: false index.search.slowlog.threshold.query.warn: 500ms index.search.slowlog.threshold.query.info: 200ms index.search.slowlog.threshold.query.debug: 100ms index.search.slowlog.threshold.query.trace: 50ms index.search.slowlog.threshold.fetch.warn: 500ms index.search.slowlog.threshold.fetch.info: 200ms index.search.slowlog.threshold.fetch.debug: 100ms index.search.slowlog.threshold.fetch.trace: 50ms index.indexing.slowlog.threshold.index.warn: 500ms index.indexing.slowlog.threshold.index.info: 200ms index.indexing.slowlog.threshold.index.debug: 1000ms index.indexing.slowlog.threshold.index.trace: 50ms script.disable_dynamic: false script.native.socialScoreCalc.type: *.SocialScriptFactory script.default_lang: native action.disable_delete_all_indices: true action.auto_create_index: .marvel-* indices.fielddata.cache.size: 30% indices.fielddata.cache.expire: 15s marvel.agent.enabled: false allow_leading_wildcard: false script.groovy.sandbox.enabled: false -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZstAUwTELHxOx%3DqWBuOjByEn3mhgdd93-kG3M%3DxQMSS7g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
elasticsearch deploy on ec2
Hi, I'm trying to deploy new elasticsearch environment on aws and the cluster is always at red and i get the following message { error: ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];], status: 503 } I'm trying to deploy 3 master nodes, 3 data nodes, and 2 client nodes. when i check cluster health: { cluster_name: stress_new, status: red, timed_out: false, number_of_nodes: 8, number_of_data_nodes: 3, active_primary_shards: 0, active_shards: 0, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0 } please adivce this is the configuration i used with changes by node role ( node.master node.data ): cluster.name: stress_new plugin.mandatory: cloud-aws cloud.aws.access_key: cloud.aws.secret_key: * cloud.aws.region: us-west-2 discovery.type: ec2 discovery.ec2.groups: stress_new_elasticsearch discovery.ec2.host_type: private_ip discovery.ec2.ping_timeout: 30s discovery.ec2.tag.elasticsearch: stress_new node.name: 172.** node.master: false node.data: false index.number_of_shards: 1 index.number_of_replicas: 0 path.data: /mnt/elasticsearch path.logs: /var/log/elasticsearch bootstrap.mlockall: true http.enabled: true gateway.recover_after_nodes: 8 gateway.expected_nodes: 8 discovery.zen.minimum_master_nodes: 3 discovery.zen.ping.timeout: 10s discovery.zen.ping.multicast.enabled: false index.search.slowlog.threshold.query.warn: 500ms index.search.slowlog.threshold.query.info: 200ms index.search.slowlog.threshold.query.debug: 100ms index.search.slowlog.threshold.query.trace: 50ms index.search.slowlog.threshold.fetch.warn: 500ms index.search.slowlog.threshold.fetch.info: 200ms index.search.slowlog.threshold.fetch.debug: 100ms index.search.slowlog.threshold.fetch.trace: 50ms index.indexing.slowlog.threshold.index.warn: 500ms index.indexing.slowlog.threshold.index.info: 200ms index.indexing.slowlog.threshold.index.debug: 1000ms index.indexing.slowlog.threshold.index.trace: 50ms script.disable_dynamic: false script.native.socialScoreCalc.type: *.SocialScriptFactory script.default_lang: native action.disable_delete_all_indices: true action.auto_create_index: .marvel-* indices.fielddata.cache.size: 30% indices.fielddata.cache.expire: 15s marvel.agent.enabled: false allow_leading_wildcard: false script.groovy.sandbox.enabled: false -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/843e39bd-842a-4301-899d-7a1bb1d119a9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:22 PM, Eliran Shlomo eli...@whipclip.com wrote: Hi, Since the first moment the cluster in in red The servers are under the same security group and inside the security group i allow any/any between the servers. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ed1d0f69-93cc-4467-a0ad-bffe4b5175df%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtJEVnyXCDkEX7Hj%3DR%3Dvyy%3DUsUiiDSXC0EVhBMU2PaYXQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Master eligible nodes and Data nodes need to have this setting -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com wrote: Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba%40googlegroups.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtzjXeHrzHiRAitfOHASpXrvzXza8JPVObtjMwCHqMrVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Remove the number of nodes setting, if that doesn't help start looking at the logs. I seen clusters on AWS that took some time to discover and stabilize, it may also be that. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:56 PM, Eliran Shlomo eli...@whipclip.com wrote: Hi, made the changes. no change in the cluster status and the same response from the servers { cluster_name: stress_new, status: red, timed_out: false, number_of_nodes: 8, number_of_data_nodes: 3, active_primary_shards: 0, active_shards: 0, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0 } get _status { error: ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];], status: 503 } On Monday, February 16, 2015 at 8:53:11 PM UTC+2, Itamar Syn-Hershko wrote: Master eligible nodes and Data nodes need to have this setting -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com wrote: Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba% 40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4Zsa0DgkCJuuLrhyrssM%2BEFMQ4jcZwECWmq%3Dq0GFyqHVEg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Hi, Since the first moment the cluster in in red The servers are under the same security group and inside the security group i allow any/any between the servers. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ed1d0f69-93cc-4467-a0ad-bffe4b5175df%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: elasticsearch deploy on ec2
Tnx for the help Itamar. solved by changing gateway.recover_after_nodes value from 8 to 6 i guess client node shouldn't be count for recover On Monday, February 16, 2015 at 8:59:48 PM UTC+2, Itamar Syn-Hershko wrote: Remove the number of nodes setting, if that doesn't help start looking at the logs. I seen clusters on AWS that took some time to discover and stabilize, it may also be that. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:56 PM, Eliran Shlomo eli...@whipclip.com javascript: wrote: Hi, made the changes. no change in the cluster status and the same response from the servers { cluster_name: stress_new, status: red, timed_out: false, number_of_nodes: 8, number_of_data_nodes: 3, active_primary_shards: 0, active_shards: 0, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0 } get _status { error: ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];], status: 503 } On Monday, February 16, 2015 at 8:53:11 PM UTC+2, Itamar Syn-Hershko wrote: Master eligible nodes and Data nodes need to have this setting -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Mon, Feb 16, 2015 at 8:31 PM, Eliran Shlomo eli...@whipclip.com wrote: Wait a second, you should use gateway.expected_data_nodes: 3 and gateway.expected_master_nodes: 3 instead of what you have there now. Also min master nodes should be 2 in your case. Those settings should be in the configuration of all nodes or only in the external gateway?(client) -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/e6914e16-28ac-4d00-96b7-b3b007538aba% 40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/cd40be93-9b10-4bf1-ac9a-eafe8b98c8fe%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9473e8bf-def6-4f04-89f3-b957394b4cf4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.