Re: time taken by each stage for a query

2014-05-23 Thread Srinivasan Ramaswamy
Thanks for the clarification. I suspected (as it was mentioned in ES site)
that significant terms feature is the bottle neck and verified the amount
of time the query takes with the feature turned on and off.  With the
feature the query takes 5s (every time) and after turning it off the query
takes only 160ms.

Does anyone have any tips on how to make significant terms feature much
faster ? Does it require a lot of JVM heap size ? Currently I am running on
a virtual machine with 16G where i allocated 8G to JVM. I have 6 nodes with
a total of 24 shards and 1 replica (default). Each shard size is 2.5G. How
can I optimize the significant terms feature ?


On Thu, May 22, 2014 at 4:39 PM, Adrien Grand 
adrien.gr...@elasticsearch.com wrote:

 That is not easy, and the reason is that Elasticsearch and Solr work in
 quite a different way eg. when it comes to compute facets/aggregations:
 Solr first computes top hits, and if facets are required, it will load the
 doc IDs of document matches into a bit set that will be used in a
 subsequent step in order to compute facets. On the other hand,
 Elasticsearch computes both top hits and facets/aggregations at the same
 time (in the same Collector if you are familiar with Lucene terminology)
 which makes timings harder to track.


 On Thu, May 22, 2014 at 6:25 AM, Srinivasan Ramaswamy 
 ursva...@gmail.comwrote:

 In the past when i used solr, i can look at time taken by each component
 to understand where most of the time is spent for a particular query.
 Similarly, I am trying to understand the breakdown of time spent for one
 particular query. Can anyone tell me how can i investigate performance of
 specific queries it in elasticsearch ?

 Thanks
 Srini

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.

 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/02f323f6-b2af-4c71-889f-9f4f7f888c57%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/02f323f6-b2af-4c71-889f-9f4f7f888c57%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.




 --
 Adrien Grand

 --
 You received this message because you are subscribed to a topic in the
 Google Groups elasticsearch group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/elasticsearch/D3ZOfKL6VKk/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5y1Md0G9WBuQxj_AokB8ro%2BuYAR9biFRKwYLdbpAkU4g%40mail.gmail.comhttps://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5y1Md0G9WBuQxj_AokB8ro%2BuYAR9biFRKwYLdbpAkU4g%40mail.gmail.com?utm_medium=emailutm_source=footer
 .

 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL1MvVzzGqK33G4hfxcai-WQocom0o5vshZgKOaoSMov9ZboDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: time taken by each stage for a query

2014-05-22 Thread Adrien Grand
That is not easy, and the reason is that Elasticsearch and Solr work in
quite a different way eg. when it comes to compute facets/aggregations:
Solr first computes top hits, and if facets are required, it will load the
doc IDs of document matches into a bit set that will be used in a
subsequent step in order to compute facets. On the other hand,
Elasticsearch computes both top hits and facets/aggregations at the same
time (in the same Collector if you are familiar with Lucene terminology)
which makes timings harder to track.


On Thu, May 22, 2014 at 6:25 AM, Srinivasan Ramaswamy ursva...@gmail.comwrote:

 In the past when i used solr, i can look at time taken by each component
 to understand where most of the time is spent for a particular query.
 Similarly, I am trying to understand the breakdown of time spent for one
 particular query. Can anyone tell me how can i investigate performance of
 specific queries it in elasticsearch ?

 Thanks
 Srini

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/02f323f6-b2af-4c71-889f-9f4f7f888c57%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/02f323f6-b2af-4c71-889f-9f4f7f888c57%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.




-- 
Adrien Grand

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j5y1Md0G9WBuQxj_AokB8ro%2BuYAR9biFRKwYLdbpAkU4g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


time taken by each stage for a query

2014-05-21 Thread Srinivasan Ramaswamy
In the past when i used solr, i can look at time taken by each component to 
understand where most of the time is spent for a particular query. 
Similarly, I am trying to understand the breakdown of time spent for one 
particular query. Can anyone tell me how can i investigate performance of 
specific queries it in elasticsearch ?

Thanks
Srini

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/02f323f6-b2af-4c71-889f-9f4f7f888c57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.