Adding shards to reduce average size
Hello I have been struggling off-and-on for months now to run an apparently highly updated ES cluster on as little hardware as possible. There's more history here http://elasticsearch-users.115913.n3.nabble.com/Sustainable-way-to-regularly-purge-deleted-docs-td4062259.html, but the overall issue is that my use case involves lots of frequent updates, which results in a growing percentage of deleted docs, and thus more RAM needed to handle the same dataset. I want to run a possible change in index structure by the group to get any possible recommendations. Ultimately the growing delete percentage seems to be related to both the size of the shards in question, and a hard-coded ratio in the Lucene Tiered Merge Policy code. As it turns out, the merge code will not consider merging a segment until the non-deleted portion of the segment is smaller than half of the max_merged_segment parameter (someone please correct me if I'm wrong here, and I wish it had been easier to discover this). This does not appear to be configurable in any version of Lucene in which I've looked. This explains why bumping max_merged_segment stupidly high can keep the deleted percentage low; the larger segments are considered for merging when they have a smaller percentage of deleted docs. This also explains why my shards grew to be ~45% deleted docs. To work around this, I'm considering reindexing with 3 or 4 times the number of shards on the same number of nodes to shrink the shards to a more manageable size, but want to make sure that the overhead from additional shards will not counter the benefits from having smaller shards. Note that I AM using routing, which I assume should reduce much of the over-sharding overhead. The index in question now has six shards with one replica each. Shard sizes ranges from 20-30 GB. Currently this is running on six r3.xlarge (30.5 GB RAM, 20GB allocated to ES) instances. Ideally I'd be running on no more than four. With a low enough delete percentage I think I can even run on three. My theory is that if I reduce the size of the shards to 5-10 GB max, then I can set the max_merged_segment high enough that the deleted docs will be merged out quickly, and I'll be able to handle the size of the merges in question without issue. With the existing setup, even on six nodes, I occasionally run into merge memory issues with large max_merged_segment sizes (13 GB currently I think). Does this seem reasonable? I'd be looking at 36-48 shards, counting replicas, being spread across 3-4 nodes, so 12-18 each. Thoughts? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/39668eef-db62-4ffc-968d-88e04f1712bd%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Sustainable way to regularly purge deleted docs
I thought I should revisit this thread in case anyone else is repeating my mistakes, which it turns out are multiple. On the bright side, I do seem to have resolved my issues. tl/dr, optimize was screwing me up, and the merge settings I thought I had in place were not actually there/active. Once applied all is well. First, the regular use of optimize?only_expunge_deletes. I did not realize at first that this command would in fact ignore the max_merged_segment parameter (I thought I had checked it at one point, but I must not have). While max_merged_segment was set to 2 GB, I ended up with segments as large as 17 GB. I reindexed everything one weekend to observe merge behaviour better and clear these out, and it wasn't until those segments were almost completely full of deleted docs that they were merged out (they finally vanished overnight, so I'm not exactly sure what the tipping point was, but I do know they were at around 4/5 deleted at one point). Clearly my use of optimize was putting the system in a state that only additional optimize calls could clean, making the cluster addicted to the optimize call. Second, and this is the more embarrassing thing, my changed merge settings had mostly not taken effect (or were reverted at some point). After removing all of the large segments via a full reindex, I added nodes to get the system to a stable point where normal merging would keep the deleted docs in check. It ended up taking 5/6 nodes to maintain ~30% delete equilibrium and enough memory to operate, which was 2-3 more nodes that I really wanted to dedicate. I decided then to bump the max_merged_segment up as per Nikolas's recommendation above (just returning it to the default 5 GB to start with), but noticed that the index merge settings were not what I thought they were. Sometime, probably months ago when I was trying to tune things originally, I apparently made a mistake, though I'm still not exactly sure when/where. I had the settings defined in the elasticsearch.yml file, but I'm guessing those are only applied to new indices when they're created, not existing indices that already have their configuration set? I know I had updated some settings via the API at some point, but perhaps I had reverted them, or simply not applied them to the index in question. Regardless, the offending index still had mostly default settings, only the max_merged_segment being different at 2 GB. I applied the settings above (plus the 5 GB max_merged_segment value) to the cluster and then performed a rolling restart to let the settings take effect. As each node came up, the deleted docs were quickly merged out of existence and the node stabilized ~3% deleted. CPU spiked to 100% while this took place, disk didn't seem to be too stressed (it reported 25% utilization when I checked via iostat at one point), but once the initial clean-up was done things settled down, and I'm expecting smaller spikes as it maintains the lower deleted percentage (I may even back down the reclaim_deletes_weight). I need to see how it actually behaves during normal load during the week before deciding everything is completely resolved, but so far things look good, and I've been able to back down to only 3 nodes again. So, I've probably wasted dozens of hours a hundreds of dollars of server time resolving what was ultimately a self-inflicted problem that should have been fixed easily months ago. So it goes. On Thursday, December 4, 2014 11:54:07 AM UTC-5, Jonathan Foy wrote: Hello I do agree with both of you that my use of optimize as regular maintenance isn't the correct way to do things, but it's been the only thing that I've found that keeps the deleted doc count/memory under control. I very much want to find something that works to avoid it. I came to much the same conclusions that you did regarding the merge settings and logic. It took a while (and eventually just reading the code) to find out that though dynamic, the merge settings don't actually take effect until a shard is moved/created (fixed in 1.4), so a lot of my early work thinking I'd changed settings wasn't really valid. That said, my merge settings are still largely what I have listed earlier in the thread, though repeating them for convenience: indices.store.throttle.type: none index.merge.policy.reclaim_deletes_weight: 6.0 -- This one I know is quite high, I kept bumping it up before I realized the changes weren't taking effect immediately index.merge.policy.max_merge_at_once: 5 index.merge.policy.max_merge_at_once_explicit: 5 index.merge.policy.segments_per_tier: 5 index.merge.policy.max_merged_segment: 2gb I DO have a mess of nested documents in the type that I know is the most troublesome...perhaps the merge logic doesn't take deleted nested documents into account when deciding what segment to merge? Or perhaps since I have a small max_merged_segment, it's like
Re: Looking for a best practice to get all data according to some filters
Just to reword what others have said, ES will allocate memory for [size] scores as I understand it (per shard?) regardless of the final result count. If you're getting back 4986 results from a query, it'd be faster to use size: 4986 than size: 100. What I've done in similar situations is to issue a count first with the same filter (which is very fast), then use the result of that in the size field. It worked much better/faster than using a default large size. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ef65d92f-9a9c-4206-a2b9-5b769e4cec87%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Sustainable way to regularly purge deleted docs
Hello I do agree with both of you that my use of optimize as regular maintenance isn't the correct way to do things, but it's been the only thing that I've found that keeps the deleted doc count/memory under control. I very much want to find something that works to avoid it. I came to much the same conclusions that you did regarding the merge settings and logic. It took a while (and eventually just reading the code) to find out that though dynamic, the merge settings don't actually take effect until a shard is moved/created (fixed in 1.4), so a lot of my early work thinking I'd changed settings wasn't really valid. That said, my merge settings are still largely what I have listed earlier in the thread, though repeating them for convenience: indices.store.throttle.type: none index.merge.policy.reclaim_deletes_weight: 6.0 -- This one I know is quite high, I kept bumping it up before I realized the changes weren't taking effect immediately index.merge.policy.max_merge_at_once: 5 index.merge.policy.max_merge_at_once_explicit: 5 index.merge.policy.segments_per_tier: 5 index.merge.policy.max_merged_segment: 2gb I DO have a mess of nested documents in the type that I know is the most troublesome...perhaps the merge logic doesn't take deleted nested documents into account when deciding what segment to merge? Or perhaps since I have a small max_merged_segment, it's like Nikolas said and those max sized segments are just rarely reclaimed in normal operation, and so the deleted doc count (and the memory they take up) grows. I don't have memory issues during normal merge operations, so I think I may start testing with a larger max segment size. I'll let you know if I ever get it resolved. On Wednesday, December 3, 2014 3:05:18 PM UTC-5, Govind Chandrasekhar wrote: Jonathan, Your current setup doesn't look ideal. As Nikolas pointed out, optimize should be run under exceptional circumstances, not for regular maintenance. That's what the merge policy setting are for, and the right settings should meet your needs, atleast theoretically. That said, I can't say I've always heeded this advice, since I've often resorted to using only_expunge_deletes when things have gotten out of hand, because it's an easy remedy to a large problem. I'm trying out a different set of settings to those Nikolas just pointed out. Since my issue is OOMs when merges take place, not so much I/O, I figured the issue is with one of two things: 1. Too many segments are being merged concurrently. 2. The size of the merged segments are large. I reduced max_merge_at_once, but this didn't fix the issue. So it had to be that the segments being merged were quite large. I noticed that my largest segments often formed 50% of each shard and had upto 30% deletes, and OOMs occurred since when these massive segments were being merged to expunge deletes, since it led to the amount of data on the shard almost doubling. To remedy this, I've REDUCED the size of max_merged_segment (I can live with more segments) and reindexed all of my data (since this doesn't help reduced existing large segments). If I understand merge settings correctly, this means that in the worst case scenario, the amount of memory used for merging will be (max_marged_segment x max_merge_at_once) GB. Since these settings don't apply retrospectively to existing large segments, I've reindexed all of my data. All of this was done in the last day or so, so I've yet to see how it works out, though I'm optimistic. By the way, I believe max_marged_segment limits are not observed for explicit optimize, so atleast in my setup, I'm going to have to shy away from explicitly expunging deletes. It could be that in your case, because of repeated explicit optimizes, or use of max_num_segments, coupled with the fact that you have a lot of reindexing going on (that too with child documents, since any change in any one of the child documents results in all other child documents and the parent document being marked as deleted), things have gotten particularly out of hand. On 3 December 2014 at 06:29, Nikolas Everett nik...@gmail.com javascript: wrote: On Wed, Dec 3, 2014 at 8:32 AM, Jonathan Foy the...@gmail.com javascript: wrote: Interesting...does the very large max_merged_segment not result in memory issues when the largest segments are merged? When I run my the cleanup command (_optimize?only_expunge_deletes) I see a steep spike in memor as each merge is completing, followed by an immediate drop, presumably as the new segment is fully initialized and then the old ones are subsequently dropped. I'd be worried that I'd run out of memory when initializing the larger segments. That being said, I only notice the large spikes when merging via the explicit optimize/only_expunge_deletes command, the continuous merging throughout the day results in very mild spikes by comparison
Re: Sustainable way to regularly purge deleted docs
Interesting...does the very large max_merged_segment not result in memory issues when the largest segments are merged? When I run my the cleanup command (_optimize?only_expunge_deletes) I see a steep spike in memor as each merge is completing, followed by an immediate drop, presumably as the new segment is fully initialized and then the old ones are subsequently dropped. I'd be worried that I'd run out of memory when initializing the larger segments. That being said, I only notice the large spikes when merging via the explicit optimize/only_expunge_deletes command, the continuous merging throughout the day results in very mild spikes by comparison. I guess I could always add a single node with the higher settings and just drop it if it becomes problematic in order to test (since, though dynamic, prior to 1.4 the merge settings only take effect on shard initialization if I remember correctly). Thanks for the advice though, I'll definitely try that. Jonathan On Tuesday, December 2, 2014 11:30:08 PM UTC-5, Nikolas Everett wrote: I've had some issues with high IO exacerbated by lots of deleted docs as well. I'd get deleted docs in the 30%-40% range on some indexes. We attacked the problem in two ways: 1. Hardware. More ram and better SSDs really really helped. No consumer grade SSDs for me. 2. Tweak some merge settings: The most important setting is index.merge.policy.max_merged_segment. You never want your settings to get near that size so set it to 30GB or something stupid huge. The way the merge policy works segments near max_merged_segment in size will end up with tons and tons of deletes before they are considered for merging and even then the merge policy will still shy away from merging them. I raised reclaim_deletes_weight slightly (2.5 or 3 or so) and lowered segments_per_tier and max_merge_at_once to get slightly better search performance. These were likely less important. I hope that helps some! Nik On Tue, Dec 2, 2014 at 9:38 PM, Jonathan Foy the...@gmail.com javascript: wrote: Hello This is something I still struggle with, though not to the degree that I once did. I've been in production for several months now with limited issues, though I still don't consider it to be a solved problem for myself, as it requires regular manual maintenance. First, I threw more hardware at it. I moved from two full time nodes to three. This helped quite a bit, and I definitely needed it once more users started hitting the cluster and more data was cached (I also added more warmers once usage patterns become more clear). Second, I've fine-tuned my sync process quite a bit to avoid unnecessary reindexing. Third, since I'm running this cluster on EC2 instances, I just spin up more nodes when I need to clean things up, then drop the number after for normal use. I had been moving to four, but now I sometimes add a fifth depending upon shard allocation - sometimes I seem to accumulate the most active shards on the same node, and I still run into memory issues. I also drop the filter cache to almost nothing before I run the optimize/delete step. For the most part, this gets me through with minimal memory issues, though I'd be screwed if I had to do this during the day. Also, there IS overhead to moving shards across nodes, and long query times when (I presume) the shards become active on the new nodes and any non-warmed fields are loaded. So, not a perfect solution by any means, but it's working. Which version of ES are you on? I'm still on 1.1.2, with plans to update soon, and am very much hoping that the update will help things become more hands-off. The bloom filter being rendered unnecessary should free memory, plus general performance improvements, I can't remember them all offhand. Being able to actually update the merge settings dynamically will also be a bit help in testing various configs. Hope something in there helps. I'm definitely open to suggestions on ways to improve things. On Tuesday, December 2, 2014 5:54:13 PM UTC-5, Govind Chandrasekhar wrote: Jonathan, Did you find a solution to this? I've been facing pretty much the same issue since I've added nested documents to my index - delete percentage goes really high and an explicit optimize leads to an OOM. Thanks. On Saturday, August 23, 2014 8:08:32 AM UTC-7, Jonathan Foy wrote: Hello I was a bit surprised to see the number of deleted docs grow so large, but I won't rule out my having something setup wrong. Non-default merge settings are below, by all means let me know if I've done something stupid: indices.store.throttle.type: none index.merge.policy.reclaim_deletes_weight: 6.0 index.merge.policy.max_merge_at_once: 5 index.merge.policy.segments_per_tier: 5 index.merge.policy.max_merged_segment: 2gb index.merge.scheduler.max_thread_count: 3 I make extensive use of nested documents
Re: Sustainable way to regularly purge deleted docs
Hello This is something I still struggle with, though not to the degree that I once did. I've been in production for several months now with limited issues, though I still don't consider it to be a solved problem for myself, as it requires regular manual maintenance. First, I threw more hardware at it. I moved from two full time nodes to three. This helped quite a bit, and I definitely needed it once more users started hitting the cluster and more data was cached (I also added more warmers once usage patterns become more clear). Second, I've fine-tuned my sync process quite a bit to avoid unnecessary reindexing. Third, since I'm running this cluster on EC2 instances, I just spin up more nodes when I need to clean things up, then drop the number after for normal use. I had been moving to four, but now I sometimes add a fifth depending upon shard allocation - sometimes I seem to accumulate the most active shards on the same node, and I still run into memory issues. I also drop the filter cache to almost nothing before I run the optimize/delete step. For the most part, this gets me through with minimal memory issues, though I'd be screwed if I had to do this during the day. Also, there IS overhead to moving shards across nodes, and long query times when (I presume) the shards become active on the new nodes and any non-warmed fields are loaded. So, not a perfect solution by any means, but it's working. Which version of ES are you on? I'm still on 1.1.2, with plans to update soon, and am very much hoping that the update will help things become more hands-off. The bloom filter being rendered unnecessary should free memory, plus general performance improvements, I can't remember them all offhand. Being able to actually update the merge settings dynamically will also be a bit help in testing various configs. Hope something in there helps. I'm definitely open to suggestions on ways to improve things. On Tuesday, December 2, 2014 5:54:13 PM UTC-5, Govind Chandrasekhar wrote: Jonathan, Did you find a solution to this? I've been facing pretty much the same issue since I've added nested documents to my index - delete percentage goes really high and an explicit optimize leads to an OOM. Thanks. On Saturday, August 23, 2014 8:08:32 AM UTC-7, Jonathan Foy wrote: Hello I was a bit surprised to see the number of deleted docs grow so large, but I won't rule out my having something setup wrong. Non-default merge settings are below, by all means let me know if I've done something stupid: indices.store.throttle.type: none index.merge.policy.reclaim_deletes_weight: 6.0 index.merge.policy.max_merge_at_once: 5 index.merge.policy.segments_per_tier: 5 index.merge.policy.max_merged_segment: 2gb index.merge.scheduler.max_thread_count: 3 I make extensive use of nested documents, and to a smaller degree child docs. Right now things are hovering around 15% deleted after a cleanup on Wednesday. I've also cleaned up my mappings a lot since I saw the 45% deleted number (less redundant data, broke some things off into child docs to maintain separately), but it was up to 30% this last weekend. When I've looked in the past when I saw the 40+% numbers, the segments in the largest tier (2 GB) would sometimes have up to 50+% deleted docs in them, the smaller segments all seemed pretty contained, which I guess makes sense as they didn't stick around for nearly as long. As for where the memory is spent, according to ElasticHQ, right now on one server I have a 20 GB heap (out of 30.5, which I know is above the 50% suggested, just trying to get things to work), I'm using 90% as follows: Field cache: 5.9 GB Filter cache: 4.0 GB (I had reduced this before the last restart, but forgot to make the changes permanent. I do use a lot of filters though, so would like to be able to use the cache). ID cache: 3.5 GB Node stats Segments: memory_in_bytes: 6.65 GB (I'm not exactly sure how this one contributes to the total heap number). As for the disk-based doc values, I don't know how I have not come across them thus far, but that sounds quite promising. I'm a little late in the game to be changing everything yet again, but it may be a good idea regardless, and is definitely something I'll read more about and consider going forward. Thank you for bringing it to my attention. Anyway, my current plan, since I'm running in AWS and have the flexibility, is just to add another r3.xlarge node to the cluster over the weekend, try the deleted-doc purge, and then pull the node back out after moving all shards off of it. I'm hoping this will allow me to clean things up with extra horsepower, but not increase costs too much throughout the week. Thanks for you input, it's very much appreciated. On Friday, August 22, 2014 7:14:18 PM UTC-4, Adrien Grand wrote: Hi Jonathan, The default merge policy is already supposed to merge quite
Native script unable to get values, perhaps because it's a child doc? ES v1.1.1
Hello I have a native script that I'm using to score/sort queries and it is not working properly for one of my three types. All three types have the same nested field, and I'm using the script to check values and score/sort by an externally defined order. However, for one of the three types the values pulled from the doc fields are always zero/null (using docFieldLongs(fieldName).getValue() or docFieldStrings(stringValue).getValue()). I can check for the fields to be present using doc().containsKey(), and it seems to see them, but it never actually sees any values. I've pulled a few records manually and verified that the data looks good. The only think I can think of that's different is that this one type is a child of one of the other two, but I'm querying it completely independently of the parent in this case. Does this sound familiar to anyone by any chance? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5fb79559-7d9f-4a32-a8e1-743876c9a152%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Native script unable to get values, perhaps because it's a child doc? ES v1.1.1
Yep, and I can search on it in other queries. After testing most of the afternoon, I finally seem to have gotten it to work by pulling the field using the full name, including the nested path: Long value = docFieldLongs(nestedPath.propertyName).getValue(); This seems to work in all three places, including the two types that also worked without the nested path, which is good. Afterwords I came across this SO post http://stackoverflow.com/questions/21289149/trouble-with-has-parent-query-containing-scripted-function-score?rq=1 which sounds like a similar problem, though I'm not in a has_parent/has_child query (though I AM in a child type). Sounds like it may be a bug. On Thursday, November 20, 2014 4:00:56 PM UTC-5, Shiwen Cheng wrote: Hi, did you index the field you want to use in the native script? Shiwen On Thursday, 20 November 2014 11:38:30 UTC-8, Jonathan Foy wrote: Hello I have a native script that I'm using to score/sort queries and it is not working properly for one of my three types. All three types have the same nested field, and I'm using the script to check values and score/sort by an externally defined order. However, for one of the three types the values pulled from the doc fields are always zero/null (using docFieldLongs(fieldName).getValue() or docFieldStrings(stringValue).getValue()). I can check for the fields to be present using doc().containsKey(), and it seems to see them, but it never actually sees any values. I've pulled a few records manually and verified that the data looks good. The only think I can think of that's different is that this one type is a child of one of the other two, but I'm querying it completely independently of the parent in this case. Does this sound familiar to anyone by any chance? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/42ed3e0c-e5a9-4673-afb6-9a65213b91c1%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Sorting by nested fields
I suspect that because you have multiple levels of nesting you need to use
the nested path
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_sorting_within_nested_objects
(it's probably choosing marketplaces.prices, and you would want just
marketplaces, though I'm not positive).
On Tuesday, October 7, 2014 11:35:37 AM UTC-4, Elke wrote:
Hi.
I have a question about sorting by nested fields.
This is just an small example of the hierarchy level (The marketplaces and
the prices are mapped as nested documents):
{ id: doc1,
name : test,
marketplaces : [ { name : marketPlace1,
prices : [ { amount: 20.00 },
{ amount: 40.00 }]
},
{ name : marketPlace2,
prices : [{ amount: 21.00 },
{ amount: 30.00 }]
}]}
{ id: doc2,
name : test2,
marketplaces : [ { name : marketPlace1,
prices : [{ amount: 30.00 },
{ amount: 35.00 }]
},
{ name : marketPlace2,
prices : [{ amount: 1.00 },
{ amount: 3.00 }]
}]}
Now I want to search for all documents with the marketplace name :
marketPlace1 and sort the result by the prices.amount of this
marketplace.
So I expect that in the example the first document will be in first place
and the second document will be in second place.
The filter works fine. I won't see the documents without marketPlace1 in
the result list.
But the sort doesn't restrict the same as the filter. So I tried to set a
nested filter to match the upper hierarchy too.
I tried this sort, but got the value Infinity as sort result. :
sort : {
marketplaces.prices.amount : { order : asc , nested_filter : {
term : {
marketplaces.name :
marketPlace1
In case I don't use the nested_filter I get the second document as first
result, because the sorting doesn't consider the marketPlace name.
Is there a solution for my problem?
Thank you very much for your help.
Best regards,
Elke
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d6a37b10-169d-40a7-9974-dc31be40bf0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Update merge settings pre-1.4 without downtime
Thanks for the response. However, you can't switch merge policies on the
fly though if I'm not mistaken; if I remember right that definitely
requires closing/reopening an index. From the docs
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-update-settings.html,
A different merge policy can't be set.
I went ahead and tested it via the index _settings API:
curl -XPUT localhost:9200/my_index/_settings -d '{
index: {
merge.policy.type: log_doc
}
}'
{error:ElasticsearchIllegalArgumentException[Can't update non dynamic
settings[[index.merge.policy.type]] for open
indices[[my_index_20140709]]],status:400}
Am I missing something?
On Friday, October 10, 2014 7:18:40 AM UTC-4, Nikolas Everett wrote:
You could try switching merge policies and then switching back. I never
tried that but I think it might work.
Nik
On Oct 10, 2014 12:33 AM, Jonathan Foy the...@gmail.com javascript:
wrote:
Hello
Is there any way of changing the merge settings of a live index without
downtime in ES versions prior to 1.4 (I'm on 1.1.1)?
After toying with some of the settings for quite a while (such as
index.merge.policy.expunge_deletes_allowed and
index.merge.policy.max_merge_at_once_explicit) I had come to the
conclusion that they were NOT updating dynamically. A dive into the code,
which also led to this issue
https://github.com/elasticsearch/elasticsearch/issues/7699, seems to
indicate that prior to 1.4 merge settings are not really dynamically
updateable, and instead would require restarting, closing/reopening an
index, or perhaps relocating shards (the updated values are stored, but
never seem to be applied). I'm looking in TieredMergePolicyProvider to get
this information.
I'm planning on trying to update our cluster to 1.4 once it's stable, but
until then it would be helpful to be able to tweak these settings as part
of my battle against deleted document creep. So, am I missing something
here, or is simply not possible right now?
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d59db65a-e467-45aa-a5bc-870c64dc8421%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d59db65a-e467-45aa-a5bc-870c64dc8421%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ec13b815-c702-4120-a4ad-3fdef79d55cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Update merge settings pre-1.4 without downtime
Hello Is there any way of changing the merge settings of a live index without downtime in ES versions prior to 1.4 (I'm on 1.1.1)? After toying with some of the settings for quite a while (such as index.merge.policy.expunge_deletes_allowed and index.merge.policy.max_merge_at_once_explicit) I had come to the conclusion that they were NOT updating dynamically. A dive into the code, which also led to this issue https://github.com/elasticsearch/elasticsearch/issues/7699, seems to indicate that prior to 1.4 merge settings are not really dynamically updateable, and instead would require restarting, closing/reopening an index, or perhaps relocating shards (the updated values are stored, but never seem to be applied). I'm looking in TieredMergePolicyProvider to get this information. I'm planning on trying to update our cluster to 1.4 once it's stable, but until then it would be helpful to be able to tweak these settings as part of my battle against deleted document creep. So, am I missing something here, or is simply not possible right now? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d59db65a-e467-45aa-a5bc-870c64dc8421%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Elastic HQ not getting back vendor info from Elasticsearch.
Development seems to have stalled on ElasticHQ for whatever reason, but look at the various forks, several people have fixed bugs, including perhaps the one you're running into. On Wednesday, August 27, 2014 10:27:41 AM UTC-4, John Smith wrote: I know, but i think it's 2 issues... 1- The vendor info for client nodes is not returned. I assume that HQ is using standard REST interface and Elasticsearch not returning the info (Maybe its a Windows SIGAR/Permission issue ???). I posted the response JSon with the bug. 2- HQ doesn't handle gracefully the fact that the info is not returned. (It's throwing JavaScript exception, but nothing shows up on the screen so you think it's frozen, when it's not) On Tuesday, 26 August 2014 19:06:35 UTC-4, Mark Walkom wrote: ElasticHQ is a community plugin, the ES devs can't help here. I have raised issues against ElasticHQ in the past and Roy has fixed them pretty quickly :) Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 27 August 2014 04:44, John Smith java.d...@gmail.com wrote: I posted an issue with Elastic HQ here: https://github.com/ royrusso/elasticsearch-HQ/issues/164 But just in case maybe an Elastic dev can have a look and see if it's Elasticsearch issue or not. Thanks -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a60bf5ec-c167-469f-b856-355faeea5601%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/a60bf5ec-c167-469f-b856-355faeea5601%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ad759da6-b00e-42d9-8396-6fe9bd4b7c5b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Optimizing queries for a 5 node cluster with 250 M documents (causes OutOfMemory exceptions and GC pauses)
I ran into the same issue when using Integer.MAX_VALUE as the size
parameter (migrating from a DB-based search). Perhaps someone can come up
with a proper reference, I cannot, but according to a comment in this SO
http://stackoverflow.com/questions/8829468/elasticsearch-query-to-return-all-records
question, Elasticsearch/Lucene tries to allocate memory for that many
scores. When I switched those queries to a count/search duo, things
improved dramatically, as you've already noticed.
On Saturday, August 23, 2014 12:17:47 PM UTC-4, Narendra Yadala wrote:
I am not returning 2 billion documents :)
I am returning all documents that match. Actual number can be anywhere
between 0 to 50k. I am just fetching documents between a given time
interval such as one hour, one day so on and then do batch processing them.
I fixed this by making 2 queries, one to fetch count and other for actual
data. It is mentioned in some other thread that scroll api is performance
intensive so I did not go for it.
On Saturday, 23 August 2014 21:32:59 UTC+5:30, Ivan Brusic wrote:
When I kept size as Integer.MAX_VALUE, it caused all the problems
Are you trying to return up to 2 billion documents at once? Even if that
number was only 1 million, you will face problems. Or did I perhaps
misunderstand you?
Are you sorting the documents based on the score (the default)?
Lucene/Elasticsearch would need to keep all the values in memory in order
to start them, causing memory problems. In general, Lucene is not effective
at deep pagination. Use scan/scroll:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html
--
Ivan
On Sat, Aug 23, 2014 at 6:46 AM, Narendra Yadala narendr...@gmail.com
wrote:
Hi Jörg,
This query
{
query : {
bool: {
must: {
match : { body : big }
},
must_not: {
match : { body : data }
},
must: {
match : {id: 521}
}
}
}
}
and this query are performing exactly same
{
query : {
bool: {
must: {
match : { body : big }
},
must_not: {
match : { body : data }
}
}
},
filter : {
term : { id : 521 }
}
}
I am not able understand what makes a filtered query fast. Is there any
place where I can find documentation on the internals of how different
queries are processed by elasticsearch.
On Saturday, 23 August 2014 18:20:23 UTC+5:30, Jörg Prante wrote:
Before firing queries, you should consider if the index design and
query choice is optimal.
Numeric range queries are not straightforward. They were a major issue
on inverted index engines like Lucene/Elasticsearch and it has taken some
time to introduce efficient implementations. See e.g.
https://issues.apache.org/jira/browse/LUCENE-1673
ES tries to compensate the downsides of massive numeric range queries
by loading all the field values into memory. To achieve effective queries,
you have to carefully discretize the values you index.
For example, a few hundred millions of different timestamps, with
millisecond resolution, are a real burden for searching on inverted
indices. A good discretization strategy for indexing is to reduce the
total
amount of values in such field to a few hundred or thousands. For
timestamps, this means, indexing time-based series data in discrete
intervals of days, hours, minutes, maybe seconds is much more efficient
than e.g. millisecond resolution.
Another topic is to use filters for boolean queries. They are much
faster.
Jörg
On Sat, Aug 23, 2014 at 2:19 PM, Narendra Yadala narendr...@gmail.com
wrote:
Hi Ivan,
Thanks for the input about aggregating on strings, I do that, but
those queries take time but they do not crash node.
The queries which caused problem were pretty straightforward queries
(such as a boolean query with two musts, one must is equal match and
other
a range match on long) but the real problem was with the size. When I
kept
size as Integer.MAX_VALUE, it caused all the problems. When I removed it,
it started working fine. I think it is worth mentioning somewhere about
this strange behavior (probably expected but strange).
I did double up on the RAM though and now I have allocated 5*10G RAM
to the cluster. Things are looking ok as of now, except that the
aggregations (on strings) are quite slow. May be I would run these
aggregations as batch and cache the outputs in a different type and move
on
for now.
Thanks
NY
On Fri, Aug 22, 2014 at 10:34 PM, Ivan Brusic iv...@brusic.com
wrote:
How expensive are your queries? Are you using aggregations or sorting
on string fields that could use up your field data cache? Are you using
the
defaults for the cache? Post the current usage.
If you
Re: Sustainable way to regularly purge deleted docs
Hello I was a bit surprised to see the number of deleted docs grow so large, but I won't rule out my having something setup wrong. Non-default merge settings are below, by all means let me know if I've done something stupid: indices.store.throttle.type: none index.merge.policy.reclaim_deletes_weight: 6.0 index.merge.policy.max_merge_at_once: 5 index.merge.policy.segments_per_tier: 5 index.merge.policy.max_merged_segment: 2gb index.merge.scheduler.max_thread_count: 3 I make extensive use of nested documents, and to a smaller degree child docs. Right now things are hovering around 15% deleted after a cleanup on Wednesday. I've also cleaned up my mappings a lot since I saw the 45% deleted number (less redundant data, broke some things off into child docs to maintain separately), but it was up to 30% this last weekend. When I've looked in the past when I saw the 40+% numbers, the segments in the largest tier (2 GB) would sometimes have up to 50+% deleted docs in them, the smaller segments all seemed pretty contained, which I guess makes sense as they didn't stick around for nearly as long. As for where the memory is spent, according to ElasticHQ, right now on one server I have a 20 GB heap (out of 30.5, which I know is above the 50% suggested, just trying to get things to work), I'm using 90% as follows: Field cache: 5.9 GB Filter cache: 4.0 GB (I had reduced this before the last restart, but forgot to make the changes permanent. I do use a lot of filters though, so would like to be able to use the cache). ID cache: 3.5 GB Node stats Segments: memory_in_bytes: 6.65 GB (I'm not exactly sure how this one contributes to the total heap number). As for the disk-based doc values, I don't know how I have not come across them thus far, but that sounds quite promising. I'm a little late in the game to be changing everything yet again, but it may be a good idea regardless, and is definitely something I'll read more about and consider going forward. Thank you for bringing it to my attention. Anyway, my current plan, since I'm running in AWS and have the flexibility, is just to add another r3.xlarge node to the cluster over the weekend, try the deleted-doc purge, and then pull the node back out after moving all shards off of it. I'm hoping this will allow me to clean things up with extra horsepower, but not increase costs too much throughout the week. Thanks for you input, it's very much appreciated. On Friday, August 22, 2014 7:14:18 PM UTC-4, Adrien Grand wrote: Hi Jonathan, The default merge policy is already supposed to merge quite aggressively segments that contain lots of deleted documents so it is a bit surprising that you can see that many numbers of deleted documents, even with merge throttling disabled. You mention having memory pressure because of the number of documents in your index, do you know what causes this memory pressure? In case it is due to field data maybe you could consider storing field data on disk? (what we call doc values) On Fri, Aug 22, 2014 at 5:27 AM, Jonathan Foy the...@gmail.com javascript: wrote: Hello I'm in the process of putting a two-node Elasticsearch cluster (1.1.2) into production, but I'm having a bit of trouble keeping it stable enough for comfort. Specifically, I'm trying to figure out the best way to keep the number of deleted documents under control. Both nodes are r3.xlarge EC2 instances (4 cores, 30.5 GB RAM). The ES cluster mirrors the primary data store, a MySQL database. Relevant updates to the database are caught via triggers which populate a table that's monitored by an indexing process. This results in what I'd consider of lot of reindexing, any time the primary data is updated. Search and indexing performance thus far has been in line with expectations when the number of deleted documents is small, but as it grows (up to 30-40%), the amount of available RAM becomes limited, ultimately causing memory problems. If I optimize/purge deletes then things return to normal, though I usually end up having to restart at least one server if not both due to OOM problems and shard failures during optimization. When ES becomes the source of all searches for the application, I can't really afford this downtime. What would be the preferred course of action here? I do have a window over the weekend where I could work with somewhat reduced capacity; I was thinking perhaps I could pull one node out of search rotation, optimize it, swap it with the other, optimize it, and then go on my way. However, I don't know that I CAN pull one node out of rotation (it seems like the search API lets me specify a node, but nothing to say Node X doesn't need any searches), nor does it appear that I can optimize an index on one node without doing the same to the other. I've tried tweaking the merge settings to favour segments containing large numbers of deletions
Sustainable way to regularly purge deleted docs
Hello I'm in the process of putting a two-node Elasticsearch cluster (1.1.2) into production, but I'm having a bit of trouble keeping it stable enough for comfort. Specifically, I'm trying to figure out the best way to keep the number of deleted documents under control. Both nodes are r3.xlarge EC2 instances (4 cores, 30.5 GB RAM). The ES cluster mirrors the primary data store, a MySQL database. Relevant updates to the database are caught via triggers which populate a table that's monitored by an indexing process. This results in what I'd consider of lot of reindexing, any time the primary data is updated. Search and indexing performance thus far has been in line with expectations when the number of deleted documents is small, but as it grows (up to 30-40%), the amount of available RAM becomes limited, ultimately causing memory problems. If I optimize/purge deletes then things return to normal, though I usually end up having to restart at least one server if not both due to OOM problems and shard failures during optimization. When ES becomes the source of all searches for the application, I can't really afford this downtime. What would be the preferred course of action here? I do have a window over the weekend where I could work with somewhat reduced capacity; I was thinking perhaps I could pull one node out of search rotation, optimize it, swap it with the other, optimize it, and then go on my way. However, I don't know that I CAN pull one node out of rotation (it seems like the search API lets me specify a node, but nothing to say Node X doesn't need any searches), nor does it appear that I can optimize an index on one node without doing the same to the other. I've tried tweaking the merge settings to favour segments containing large numbers of deletions, but it doesn't seem to make enough of a difference. I've also disabled merge throttling (I do have SSD-backed storage). Is there any safe way to perform regular maintenance on the cluster, preferably one node at a time, without causing TOO many problems? Am I just trying to do too much with the hardware I have? Any advice is appreciated. Let me know what info I left out that would help. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/65b96db1-0e56-4681-b73d-c21365983199%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Every other query slow
Note also that the slow instances of the query do not appear to show up in
the slow query log. Also, I'm pulling the referenced times out of the
response's took field.
On Tuesday, July 8, 2014 11:09:21 PM UTC-4, Jonathan Foy wrote:
Hello
I'm trying to get a new ES cluster tuned properly to actually put into
production, and I'm running into some performance issues.
While testing, I noticed that when running the same query multiple times,
I had alternating fast (~50 ms), and slow (2-3 s) results. It's the exact
same query, submitted via curl, and it happens consistently query after
query.
curl -XGET localhost:9200/my_index/my_type/_search?routing=1234 -d
@fileWithQuery
I literally hit up/enter time after time. At one point I wandered away
for ~30 minutes after a slow execution, came back, up/enter, it finished in
40 ms. The immediate next attempt, 2 seconds.
I'm running ES 1.1.1 on a two-node cluster. There's three indexes, three
shards each for the smaller two, six shards for the larger index, which is
the one I'm hitting. I'm using custom routing for all three. One replica
of each, so all 12 shards on each server. The index in question is ~125
GB. The other two are 10 GB and 2 GB more or less.
Summary server info:
ES 1.1.1
2 AWS r3.xlarge instances (30.5 GB each)
One 18 GB heap/G1 GC
One 20 GB heap/default GC
I know the heap is set higher than recommended, but I wouldn't think
that'd be the current problem.
My first thought was that I was simply hitting one server and then the
other via round-robin, and I needed to figure out which server was slow.
However, the stats reported in ElasticHQ indicated that the queries were
hitting the same server each time (there was no other searching going on
and limited indexing). Even when I tried running the search from the other
server, ElasticHQ still indicated that they queries were running on the one
server (and the same fast/slow/fast pattern was noticed, though independent
of the cycle on the other server). I'm not sure why the other server was
never being hit, though ElasticHQ DID report about 3x the amount of search
activity on the server on which the queries were running. That might be my
next question.
There are warmers in place for some of the fields. Field cache reports no
evictions and hovers around 4GB, though there ARE a lot of evictions in the
filter cache. I think that's probably inevitable given how much variety
can come through in the searches, though I'm open to advice.
I've pasted a sample query below. It's admittedly a bit ugly, because
it's built dynamically from a large number of search criteria with various
levels of nesting. I've tried cleaned up versions of the same query
(remove unnecessary filters) with the same results, but included it as is
(with renamed fields) in case there's something wrong.
Note that while I've been testing and writing this post, I found that
removing the nested sort and instead sorting on a non-nested field does not
result in the fast/slow/fast pattern, they're all fast. However, I've
since tested other queries, including some with no sort/limit at all, and
found the same pattern. There is a lot of nesting, and sometimes has_child
filters. Executing somewhat different (though admittedly similar) queries
results in the same pattern across queries, regardless of which is run
when. Fast/slow/fast.
So, any idea as to what is going on here? The fast queries are completely
adequate, the slow queries completely inadequate. I need to figure this
out.
Let me know if any other info is needed. Thanks in advance.
{
from : 0,
size : 50,
query : {
filtered : {
query : {
match_all : { }
},
filter : {
and : {
filters : [ {
term : {
accountId : 1234
}
}, {
nested : {
filter : {
and : {
filters : [ {
nested : {
filter : {
and : {
filters : [ {
or : {
filters : [ {
term : {
stage1.stage2.bool1 : true
}
}, {
term : {
stage1.stage2.bool2 : false
}
} ]
}
} ]
}
},
path : stage1.stage2
}
} ]
}
},
path : stage1
}
} ]
}
}
}
},
fields : id,
sort
Every other query slow
Hello
I'm trying to get a new ES cluster tuned properly to actually put into
production, and I'm running into some performance issues.
While testing, I noticed that when running the same query multiple times, I
had alternating fast (~50 ms), and slow (2-3 s) results. It's the exact
same query, submitted via curl, and it happens consistently query after
query.
curl -XGET localhost:9200/my_index/my_type/_search?routing=1234 -d
@fileWithQuery
I literally hit up/enter time after time. At one point I wandered away for
~30 minutes after a slow execution, came back, up/enter, it finished in 40
ms. The immediate next attempt, 2 seconds.
I'm running ES 1.1.1 on a two-node cluster. There's three indexes, three
shards each for the smaller two, six shards for the larger index, which is
the one I'm hitting. I'm using custom routing for all three. One replica
of each, so all 12 shards on each server. The index in question is ~125
GB. The other two are 10 GB and 2 GB more or less.
Summary server info:
ES 1.1.1
2 AWS r3.xlarge instances (30.5 GB each)
One 18 GB heap/G1 GC
One 20 GB heap/default GC
I know the heap is set higher than recommended, but I wouldn't think that'd
be the current problem.
My first thought was that I was simply hitting one server and then the
other via round-robin, and I needed to figure out which server was slow.
However, the stats reported in ElasticHQ indicated that the queries were
hitting the same server each time (there was no other searching going on
and limited indexing). Even when I tried running the search from the other
server, ElasticHQ still indicated that they queries were running on the one
server (and the same fast/slow/fast pattern was noticed, though independent
of the cycle on the other server). I'm not sure why the other server was
never being hit, though ElasticHQ DID report about 3x the amount of search
activity on the server on which the queries were running. That might be my
next question.
There are warmers in place for some of the fields. Field cache reports no
evictions and hovers around 4GB, though there ARE a lot of evictions in the
filter cache. I think that's probably inevitable given how much variety
can come through in the searches, though I'm open to advice.
I've pasted a sample query below. It's admittedly a bit ugly, because it's
built dynamically from a large number of search criteria with various
levels of nesting. I've tried cleaned up versions of the same query
(remove unnecessary filters) with the same results, but included it as is
(with renamed fields) in case there's something wrong.
Note that while I've been testing and writing this post, I found that
removing the nested sort and instead sorting on a non-nested field does not
result in the fast/slow/fast pattern, they're all fast. However, I've
since tested other queries, including some with no sort/limit at all, and
found the same pattern. There is a lot of nesting, and sometimes has_child
filters. Executing somewhat different (though admittedly similar) queries
results in the same pattern across queries, regardless of which is run
when. Fast/slow/fast.
So, any idea as to what is going on here? The fast queries are completely
adequate, the slow queries completely inadequate. I need to figure this
out.
Let me know if any other info is needed. Thanks in advance.
{
from : 0,
size : 50,
query : {
filtered : {
query : {
match_all : { }
},
filter : {
and : {
filters : [ {
term : {
accountId : 1234
}
}, {
nested : {
filter : {
and : {
filters : [ {
nested : {
filter : {
and : {
filters : [ {
or : {
filters : [ {
term : {
stage1.stage2.bool1 : true
}
}, {
term : {
stage1.stage2.bool2 : false
}
} ]
}
} ]
}
},
path : stage1.stage2
}
} ]
}
},
path : stage1
}
} ]
}
}
}
},
fields : id,
sort : [ {
website.domain.sortable : {
order : asc,
missing : 0,
nested_path : website
}
}, {
id : {
order : asc
}
} ]
}
--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
Warmer queries - many small or one large
Hello The subject pretty much says it all...is there an advantage one way or the other to having several (or many) small (single term) warmer queries rather than a single large query that searches all desired fields? -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/eb396abd-514d-4d81-9488-aa0fde98de18%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Sorting by a nested multi-field only works for one of the fields
Hello In my mapping I have some String values that I need to use for text matching as well as for sorting. To support this, I use a multi-field with two different versions of the field using two different analyzers. To sort, I have a keyword-lowercasing analyzer. For the text search I'm using the included snowball analyzer. This works fine for a normal, top-level field. However, for my nested fields using this pattern, only the first/primary field type will allow searching. The output from tests using the other always returns a null value. I'm assuming it's somehow related to the pathing (mixing nested pathing notation with multi-field notation), but I'm not sure. Does anyone have any insight? I have a full working example herehttp://sense.qbox.io/gist/92524e522f5096f98c59cc6f2da7ffa213433536 . -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cd3d5501-a34a-4a83-88b6-d5165cec77f9%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Sorting by a nested multi-field only works for one of the fields
On Thursday, April 24, 2014 8:12:26 AM UTC-4, Jonathan Foy wrote: Hello In my mapping I have some String values that I need to use for text matching as well as for sorting. To support this, I use a multi-field with two different versions of the field using two different analyzers. To sort, I have a keyword-lowercasing analyzer. For the text search I'm using the included snowball analyzer. This works fine for a normal, top-level field. However, for my nested fields using this pattern, only the first/primary field type will allow searching. The output from tests using the other always returns a null value. I'm assuming it's somehow related to the pathing (mixing nested pathing notation with multi-field notation), but I'm not sure. Does anyone have any insight? I have a full working example herehttp://sense.qbox.io/gist/92524e522f5096f98c59cc6f2da7ffa213433536 . Received help on SOhttp://stackoverflow.com/questions/23255991/elasticsearch-sorting-by-a-nested-multi-field#23275061, I was missing the nested_path parameter in the sort definition. -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/284b0c4a-d570-4e07-9201-c30a38f4e63d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
