This is something that I just "discovered" as well. Using a top-level filter is really a "post_filter" (it's renamed in later versions of ES): http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-post-filter.html
So, this will execute the query first (a default "match_all: {}") and then execute the filter on that result set. This is not very efficient for your query, since I expect you expected having filter there to work like a "pre-filter" and filter out results *before* executing the query. To do that, you need to use a "filtered query": http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-filtered-query.html In your case, the resulting query would look like: curl -XGET 'http://localhost:9200/my-index/my-doc-type/_search' -d '{ "query": { "filtered": { "filter": { "term": {"void": false} } } }, "fields": [["user_id1", "user_name", "date", "status", "q1", "q1_unique_code", "q2", "q3"]], "size": 50000, "sort": ["date_value"]}' On Fri, Nov 21, 2014 at 7:07 AM, Ajay Divakaran <ajay.divakara...@gmail.com> wrote: > The term filter that is used: > > curl -XGET 'http://localhost:9200/my-index/my-doc-type/_search' -d '{ > "filter": { > "term": {"void": false} > }, > "fields": [["user_id1", "user_name", "date", "status", "q1", > "q1_unique_code", "q2", "q3"]], > "size": 50000, "sort": ["date_value"]}' > > > - The 'void' field is a boolean field. > - The index store size is 504mb. > - The elastic search setup consists of only a single node and the > index consists of only a single shard and 0 replicas. The version of > elasticsearch is 0.90.7 > - The fields mentioned above is only the first 8 fields. The actual > term filter that we execute has 350 fields mentioned. > > *We noticed the memory spiking by about 2-3gb though the store size is > only 504mb.* > > *Running the query multiple times seems to continuously increase the > memory.* > > Could someone explain why this memory spike occurs? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/7c4ea660-9411-4d1d-a86c-84f1c43f4f7e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Nick Canzoneri Developer, Wildbit <http://wildbit.com/> Beanstalk <http://beanstalkapp.com/>, Postmark <http://postmarkapp.com/>, dploy.io -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKWm5yPQo5G0w2ShxtC4E9K90Bji1tQca3RqcG%2BvGUDR8aAqMQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.