Indeed that is what I meant.
On Fri, Jun 13, 2014 at 10:33 AM, Thomas <thomas.bo...@gmail.com> wrote: > So I restructured my curl as follows, is this what you mean?, by doing > some first hits i do get some slight improvement, but need to check into > production data: > > Thank you will try it and come back with results > > curl -XPOST " > http://10.129.2.42:9200/logs-idx.20140613/event/_search?search_type=count"; > -d' > { > "query": { > "filtered": { > > "filter": { > "or": [ > { > "and": [ > { > "has_parent": { > "type": "request", > "filter": { > "and": { > "filters": [ > { > "term": { > "country": "US" > } > }, > { > "term": { > "city": "NY" > } > }, > { > "term": { > "code": 12 > } > } > ] > } > } > } > }, > { > "range": { > "event_time": { > "gte": "2014-06-13T10:00:00", > "lt": "2014-06-13T11:00:00" > } > } > } > ] > }, > { > "and": [ > { > "has_parent": { > "type": "request", > "filter": { > "and": { > "filters": [ > { > "term": { > "country": "US" > } > }, > { > "term": { > "city": "NY" > } > }, > { > "term": { > "code": 12 > } > }, > { > "range": { > "request_time": { > "gte": "2014-06-13T10:00:00", > "lt": "2014-06-13T11:00:00" > } > } > } > ] > } > } > } > }, > { > "range": { > "event_time": { > "lt": "2014-06-13T10:00:00" > } > } > } > ] > } > ] > } > } > }, > "aggs": { > "per_interval": { > "date_histogram": { > "field": "event_time", > "interval": "minute" > }, > "aggs": { > "metrics": { > "terms": { > "field": "event", > "size": 12 > } > } > } > } > } > }' > > > On Friday, 13 June 2014 10:09:46 UTC+3, Thomas wrote: > >> Hi, >> >> I'm facing a performance issue with some aggregations I perform, and I >> need your help if possible: >> >> I have to documents, the *request* and the *event*. The request is the >> parent of the event. Below is a (sample) mapping >> >> "event" : { >> "dynamic" : "strict", >> "_parent" : { >> "type" : "request" >> }, >> "properties" : { >> "event_time" : { >> "format" : "dateOptionalTime", >> "type" : "date" >> }, >> "count" : { >> "type" : "integer" >> }, >> "event" : { >> "index" : "not_analyzed", >> "type" : "string" >> } >> } >> } >> >> "request" : { >> "dynamic" : "strict", >> "_id" : { >> "path" : "uniqueId" >> }, >> "properties" : { >> "uniqueId" : { >> "index" : "not_analyzed", >> "type" : "string" >> }, >> "user" : { >> "index" : "not_analyzed", >> "type" : "string" >> }, >> "code" : { >> "type" : "integer" >> }, >> "country" : { >> "index" : "not_analyzed", >> "type" : "string" >> }, >> "city" : { >> "index" : "not_analyzed", >> "type" : "string" >> } >> .... >> } >> } >> >> My cluster is becoming really big (almost 2 TB of data with billions of >> documents) and i maintain one index per day, whereas I occasionally delete >> old indices. My daily index is about 20GB big. The version of elasticsearch >> that I use is 1.1.1. >> >> My problems start when I want to get some aggregations of events with >> some criteria which is applied in the parent request document. For example >> count be the events of type *click for country = US and code=12. What I >> was initially doing was to generate a scriptFilter for the request document >> (in Groovy) and I was adding multiple aggregations in one search request. >> This ended up being very slow so I removed the scripting logic and I >> supported my logic with java code.* >> >> What seems to be initially solved in my local machine, when I got back to >> the cluster, nothing has changed. Again my app performs really really poor. >> I get more than 10 seconds to perform a search with ~10 sub-aggregations. >> >> What seems strange is that I notice that the cluster is pretty ok with >> regards load average, CPU etc. >> >> Any hints on where to look for solving this out? to be able to identify >> the bottleneck >> >> *Ask for any additional information to provide*, I didn't want to make >> this post too long to read >> Thank you >> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/2d8da0eb-0700-4bea-9acb-b8052db77e05%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/2d8da0eb-0700-4bea-9acb-b8052db77e05%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7hBkKROnL%3D7mTFua6jPtnXTHY%3D0iqRkBdCmNePd1OaZA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
