shard allocation per node
Hi, What is the rule for primary shard allocation for single index? I created one index with 9 primary and 0 replica shards. Elasticsearch allocated 5 primary shards on ES01 server (the node with least storage available) and 4 more shards on different nodes. I have 9 servers. For this index only 5 servers are in use. I don't think this is a correct behavior . The free space is pretty much even on every host, except ES01 (i still don't understand why), and i did't hit any hi/low watermarks. Any idea what is happening here ? Georgi -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2d5f1140-cb10-4ac4-8bd7-2a41433e61f4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disk-based shard allocation per node settings
If I hava multiple data dirs on a node, can I apply these settings to each directory individually? On Friday, May 16, 2014 9:31:11 AM UTC-4, Itamar Syn-Hershko wrote: The link you provided is for transient settings, meaning they will reset once the cluster restarts. Doing this via the configurations file ensures the settings are loaded back. As Mark said, this is cluster level settings. If you have nodes with different HD characteristics you probably want to use percentages for the watermark values. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, May 16, 2014 at 3:26 PM, Michel Conrad michel...@trendiction.com javascript: wrote: What if I set the option in the configuration file, will it then only be applied to the node reading the configuration file? On Fri, May 16, 2014 at 10:36 AM, Mark Walkom ma...@campaignmonitor.com javascript: wrote: It's a cluster level setting, so all nodes use the same one. As far as I know, there is no node level setting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com javascript: web: www.campaignmonitor.com On 16 May 2014 17:43, Michel Conrad michel...@trendiction.com javascript: wrote: Is it possible to specify different thresholds for different nodes in the cluster? I could not find it in the documentation, when updating the setting through the cluster settings as described on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk the threshold is applied to every node. When specifing the thresholds through the configuration file, will it then be only applied to the node reading the configuration? In this case how to I prevent the cluster settings being applied and overwriting the configuration file (remove a setting). Thanks for any thoughts, Michel -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYgM4%3DmH2cA3wJoBQAmx82ddrz6N%3D8u3XE0EFoKoOwTUUQ%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624a6tk9jqGkh_ED%3Dx_bpJPxpiydP9_KxTugrBVZuX5yaCw%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYjc5DAO4mW%2B%3DnrgDfxbEd%2BYxwcn1vZ9jXxBNm1N%2BH9%3D1Q%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/63097500-b95f-49a4-a5de-190c15ecdb32%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Disk-based shard allocation per node settings
Nope, it's for the node as a whole. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 3 October 2014 07:18, Andrew Mehler meh...@gmail.com wrote: If I hava multiple data dirs on a node, can I apply these settings to each directory individually? On Friday, May 16, 2014 9:31:11 AM UTC-4, Itamar Syn-Hershko wrote: The link you provided is for transient settings, meaning they will reset once the cluster restarts. Doing this via the configurations file ensures the settings are loaded back. As Mark said, this is cluster level settings. If you have nodes with different HD characteristics you probably want to use percentages for the watermark values. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, May 16, 2014 at 3:26 PM, Michel Conrad michel...@trendiction.com wrote: What if I set the option in the configuration file, will it then only be applied to the node reading the configuration file? On Fri, May 16, 2014 at 10:36 AM, Mark Walkom ma...@campaignmonitor.com wrote: It's a cluster level setting, so all nodes use the same one. As far as I know, there is no node level setting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 May 2014 17:43, Michel Conrad michel...@trendiction.com wrote: Is it possible to specify different thresholds for different nodes in the cluster? I could not find it in the documentation, when updating the setting through the cluster settings as described on http://www.elasticsearch.org/guide/en/elasticsearch/ reference/current/index-modules-allocation.html#disk the threshold is applied to every node. When specifing the thresholds through the configuration file, will it then be only applied to the node reading the configuration? In this case how to I prevent the cluster settings being applied and overwriting the configuration file (remove a setting). Thanks for any thoughts, Michel -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYgM4% 3DmH2cA3wJoBQAmx82ddrz6N%3D8u3XE0EFoKoOwTUUQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ CAEM624a6tk9jqGkh_ED%3Dx_bpJPxpiydP9_KxTugrBVZuX5yaCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/ msgid/elasticsearch/CAH0sEYjc5DAO4mW%2B%3DnrgDfxbEd% 2BYxwcn1vZ9jXxBNm1N%2BH9%3D1Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/63097500-b95f-49a4-a5de-190c15ecdb32%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/63097500-b95f-49a4-a5de-190c15ecdb32%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624baF4J1DxJBE_X%2BoCg29NG0jrnFZGzPXoTmMt6F9sqHsg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Disk-based shard allocation per node settings
It's a cluster level setting, so all nodes use the same one. As far as I know, there is no node level setting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 May 2014 17:43, Michel Conrad michel.con...@trendiction.com wrote: Is it possible to specify different thresholds for different nodes in the cluster? I could not find it in the documentation, when updating the setting through the cluster settings as described on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk the threshold is applied to every node. When specifing the thresholds through the configuration file, will it then be only applied to the node reading the configuration? In this case how to I prevent the cluster settings being applied and overwriting the configuration file (remove a setting). Thanks for any thoughts, Michel -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYgM4%3DmH2cA3wJoBQAmx82ddrz6N%3D8u3XE0EFoKoOwTUUQ%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624a6tk9jqGkh_ED%3Dx_bpJPxpiydP9_KxTugrBVZuX5yaCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Disk-based shard allocation per node settings
What if I set the option in the configuration file, will it then only be applied to the node reading the configuration file? On Fri, May 16, 2014 at 10:36 AM, Mark Walkom ma...@campaignmonitor.com wrote: It's a cluster level setting, so all nodes use the same one. As far as I know, there is no node level setting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 May 2014 17:43, Michel Conrad michel.con...@trendiction.com wrote: Is it possible to specify different thresholds for different nodes in the cluster? I could not find it in the documentation, when updating the setting through the cluster settings as described on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk the threshold is applied to every node. When specifing the thresholds through the configuration file, will it then be only applied to the node reading the configuration? In this case how to I prevent the cluster settings being applied and overwriting the configuration file (remove a setting). Thanks for any thoughts, Michel -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYgM4%3DmH2cA3wJoBQAmx82ddrz6N%3D8u3XE0EFoKoOwTUUQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624a6tk9jqGkh_ED%3Dx_bpJPxpiydP9_KxTugrBVZuX5yaCw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYjc5DAO4mW%2B%3DnrgDfxbEd%2BYxwcn1vZ9jXxBNm1N%2BH9%3D1Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Disk-based shard allocation per node settings
The link you provided is for transient settings, meaning they will reset once the cluster restarts. Doing this via the configurations file ensures the settings are loaded back. As Mark said, this is cluster level settings. If you have nodes with different HD characteristics you probably want to use percentages for the watermark values. -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action http://manning.com/synhershko/ On Fri, May 16, 2014 at 3:26 PM, Michel Conrad michel.con...@trendiction.com wrote: What if I set the option in the configuration file, will it then only be applied to the node reading the configuration file? On Fri, May 16, 2014 at 10:36 AM, Mark Walkom ma...@campaignmonitor.com wrote: It's a cluster level setting, so all nodes use the same one. As far as I know, there is no node level setting. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 16 May 2014 17:43, Michel Conrad michel.con...@trendiction.com wrote: Is it possible to specify different thresholds for different nodes in the cluster? I could not find it in the documentation, when updating the setting through the cluster settings as described on http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk the threshold is applied to every node. When specifing the thresholds through the configuration file, will it then be only applied to the node reading the configuration? In this case how to I prevent the cluster settings being applied and overwriting the configuration file (remove a setting). Thanks for any thoughts, Michel -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYgM4%3DmH2cA3wJoBQAmx82ddrz6N%3D8u3XE0EFoKoOwTUUQ%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624a6tk9jqGkh_ED%3Dx_bpJPxpiydP9_KxTugrBVZuX5yaCw%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAH0sEYjc5DAO4mW%2B%3DnrgDfxbEd%2BYxwcn1vZ9jXxBNm1N%2BH9%3D1Q%40mail.gmail.com . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtTZfb%2B0uJSxohd685pCAcEgxZXSg_SYeu9izq%3D4_1zjQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
