Re: Non-Uniform Drive Space Across Nodes
OP here. My numbers on the disk space were not an actual observation of current sizes. It was more of a hypothetical of what can I expect ES to do if I only had three servers and that was the starting disk space available in each. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca665ae3-393e-4f79-8a31-60130751938f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Non-Uniform Drive Space Across Nodes
How does Elasticsearch handle nodes that do not have the same amount of disk space on each node. Looking at the example below does ES limit storage to that of the smaller one or does it make use of all the space just allocating the shards accordingly? Or is it even smart enough to take disk space into consideration at all? Node 1 - 1 TB Node 2 - 750 GB Node 3 - 500 GB -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Non-Uniform Drive Space Across Nodes
Yes, take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk Although the docs mention coming in 1.3, I could have sworn it was still available for 1.2. On a general note, ES should distribute things evenly across all nodes, however I have noticed similar things on some of our nodes. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 12 June 2014 22:27, ES USER es.user.2...@gmail.com wrote: How does Elasticsearch handle nodes that do not have the same amount of disk space on each node. Looking at the example below does ES limit storage to that of the smaller one or does it make use of all the space just allocating the shards accordingly? Or is it even smart enough to take disk space into consideration at all? Node 1 - 1 TB Node 2 - 750 GB Node 3 - 500 GB -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Non-Uniform Drive Space Across Nodes
The disk threshold stuff has been available since sometime in 0.90. In 1.3 it'll be on by default. It works by stopping allocation to shards with disks over a watermark and moving shards off nodes that are over another higher mark. Meaning elasticsearch won't try to balance usage. Just keep from filling up disk. It's a somewhat fine distinction but a real one. On Jun 12, 2014 6:37 PM, Mark Walkom ma...@campaignmonitor.com wrote: Yes, take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk Although the docs mention coming in 1.3, I could have sworn it was still available for 1.2. On a general note, ES should distribute things evenly across all nodes, however I have noticed similar things on some of our nodes. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 12 June 2014 22:27, ES USER es.user.2...@gmail.com wrote: How does Elasticsearch handle nodes that do not have the same amount of disk space on each node. Looking at the example below does ES limit storage to that of the smaller one or does it make use of all the space just allocating the shards accordingly? Or is it even smart enough to take disk space into consideration at all? Node 1 - 1 TB Node 2 - 750 GB Node 3 - 500 GB -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1zx7hNzfgd-ZN%3Doq-VzGUCUXXNr%3D5Bc2x9uNL9mVMNgw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Non-Uniform Drive Space Across Nodes
What's the ideal way to diagnose the initial problem stated? I can see in our cluster we have a node with 43GB (of 500) free, yet most others have around 100GB free (of 500). I can see the shard count per node (using _cat/shards) seems to roughly the same, but could it be because some of our shards are just different sizes? In the case of the OP though, a 500GB difference seems difficult to explain. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 13 June 2014 09:00, Nikolas Everett nik9...@gmail.com wrote: The disk threshold stuff has been available since sometime in 0.90. In 1.3 it'll be on by default. It works by stopping allocation to shards with disks over a watermark and moving shards off nodes that are over another higher mark. Meaning elasticsearch won't try to balance usage. Just keep from filling up disk. It's a somewhat fine distinction but a real one. On Jun 12, 2014 6:37 PM, Mark Walkom ma...@campaignmonitor.com wrote: Yes, take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk Although the docs mention coming in 1.3, I could have sworn it was still available for 1.2. On a general note, ES should distribute things evenly across all nodes, however I have noticed similar things on some of our nodes. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 12 June 2014 22:27, ES USER es.user.2...@gmail.com wrote: How does Elasticsearch handle nodes that do not have the same amount of disk space on each node. Looking at the example below does ES limit storage to that of the smaller one or does it make use of all the space just allocating the shards accordingly? Or is it even smart enough to take disk space into consideration at all? Node 1 - 1 TB Node 2 - 750 GB Node 3 - 500 GB -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1zx7hNzfgd-ZN%3Doq-VzGUCUXXNr%3D5Bc2x9uNL9mVMNgw%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1zx7hNzfgd-ZN%3Doq-VzGUCUXXNr%3D5Bc2x9uNL9mVMNgw%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZZz5NyYYHVg8OW6JEV4vkgKNnK28wtWiWOX5kD05Pzeg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Non-Uniform Drive Space Across Nodes
I'm not sure what problem you mean. If you mean that you have uneven disk utilization then I don't think there is something for that. You could raise the index weight in the allocation weights. That'd spread the shards of each index out more evenly. It might help. On Jun 12, 2014 7:26 PM, Mark Walkom ma...@campaignmonitor.com wrote: What's the ideal way to diagnose the initial problem stated? I can see in our cluster we have a node with 43GB (of 500) free, yet most others have around 100GB free (of 500). I can see the shard count per node (using _cat/shards) seems to roughly the same, but could it be because some of our shards are just different sizes? In the case of the OP though, a 500GB difference seems difficult to explain. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 13 June 2014 09:00, Nikolas Everett nik9...@gmail.com wrote: The disk threshold stuff has been available since sometime in 0.90. In 1.3 it'll be on by default. It works by stopping allocation to shards with disks over a watermark and moving shards off nodes that are over another higher mark. Meaning elasticsearch won't try to balance usage. Just keep from filling up disk. It's a somewhat fine distinction but a real one. On Jun 12, 2014 6:37 PM, Mark Walkom ma...@campaignmonitor.com wrote: Yes, take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk Although the docs mention coming in 1.3, I could have sworn it was still available for 1.2. On a general note, ES should distribute things evenly across all nodes, however I have noticed similar things on some of our nodes. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 12 June 2014 22:27, ES USER es.user.2...@gmail.com wrote: How does Elasticsearch handle nodes that do not have the same amount of disk space on each node. Looking at the example below does ES limit storage to that of the smaller one or does it make use of all the space just allocating the shards accordingly? Or is it even smart enough to take disk space into consideration at all? Node 1 - 1 TB Node 2 - 750 GB Node 3 - 500 GB -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/8ed0d604-a8b4-41a0-924d-7d8af030fcd5%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAEM624Yqo5YELaNzhZzYCiAfFEU-fRdM-Xsfn9AT-K-uLCpiuA%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1zx7hNzfgd-ZN%3Doq-VzGUCUXXNr%3D5Bc2x9uNL9mVMNgw%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1zx7hNzfgd-ZN%3Doq-VzGUCUXXNr%3D5Bc2x9uNL9mVMNgw%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZZz5NyYYHVg8OW6JEV4vkgKNnK28wtWiWOX5kD05Pzeg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAEM624ZZz5NyYYHVg8OW6JEV4vkgKNnK28wtWiWOX5kD05Pzeg%40mail.gmail.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the
