On Wed, Dec 3, 2014 at 8:32 AM, Jonathan Foy <the...@gmail.com> wrote:
Interesting...does the very large max_merged_segment not result in memory > issues when the largest segments are merged? When I run my the cleanup > command (_optimize?only_expunge_deletes) I see a steep spike in memor as > each merge is completing, followed by an immediate drop, presumably as the > new segment is fully initialized and then the old ones are subsequently > dropped. I'd be worried that I'd run out of memory when initializing the > larger segments. That being said, I only notice the large spikes when > merging via the explicit optimize/only_expunge_deletes command, the > continuous merging throughout the day results in very mild spikes by > comparison. > I don't see memory issues but I'm not really looking for them. Memory usage has never been a problem for us. IO spikes were a problem the few times I ran only_expunge_deletes. I'm forming the opinion that calling _optimize is should be a pretty remarkable thing. Like it should only be required when: 1. You are done writing an index and will never touch it again and want to save some space/make querying a bit faster. 2. You are working around some funky bug. 3. You've just built the index with funky merge settings that created a bazillion segments but imported quickly. 4. You shouldn't be calling it. Stop now. You've made a mistake. I think that #1 and #3 aren't valid for only_expunge_deletes though. So that leaves either - you are working around a bug or you are making a mistake. In your case I _think_ your mistake is taking the default merge settings. Maybe. Or maybe that is a bug. I'm not sure. If it is a mistake then you are in good company. Also! only_expunge_deletes is kind of a trappy name - what it really does is smash all the segments with deletes together into one big segment making the max_merged_segment worse in the long run. A steep spike in memory usage is probably not worth worrying about so long as you don't see any full GCs done via stop the world (concurrent mode failure). I'd expect to see more minor GCs during the spike and those are stop the world but they _should_ be pretty short. Elasticsearch should log a WARNING or ERROR during concurrent mode failures. It also exposes counters of all the time spent in minor and full GCs and you can jam those into RRDtool to get some nice graphs. Marvel will probably do that for you, I'm not sure. You can also use `jstat -gcutil <pid> 1s 10000` to get it to spit out the numbers in real time. > > I guess I could always add a single node with the higher settings and just > drop it if it becomes problematic in order to test (since, though dynamic, > prior to 1.4 the merge settings only take effect on shard initialization if > I remember correctly). > I'm pretty sure that is an index level settings. Also, I think there was an issue with applying it live in some versions but I know its fixed in 1.4. I'm pretty sure you can trick your way around the issue by moving the shard to another node. Its kind of fun. > Thanks for the advice though, I'll definitely try that. > > Good Luck! Nik -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd3LXVJ4O2pZZjz3hDH9w499GOQ85mAsk-TLp0Y3E8YC2Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
