High fetch times

2014-01-15 Thread Arik Gaisler
Im using ElasticHQ to monitor our cluster and I am noticing that our Search Fetch times are about 5X more then query times, which seems unreasonable (20ms vs ~4ms) Im also noticing a 1.44 mb swap space on one of the nodes (the other node is at 0) We're using a 2 node cluster, 2 shards 1

Re: High fetch times

2014-01-15 Thread Itamar Syn-Hershko
What is the query type you are using? http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Author of RavenDB in Action

Re: High fetch times

2014-01-15 Thread Arik Gaisler
We are doing a query then fetch serach type On Wednesday, January 15, 2014 4:48:58 PM UTC+2, Itamar Syn-Hershko wrote: What is the query type you are using? http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html -- Itamar Syn-Hershko

Re: High fetch times

2014-01-15 Thread Itamar Syn-Hershko
I believe this makes sense then - search is entirely in-memory operation (after a certain warm up), and fetch involves disk IO. Try keep your docs as small as possible and don't disable _source, but other than that you're bounded by the speed of your disks. -- Itamar Syn-Hershko

Re: High fetch times

2014-01-15 Thread Itamar Syn-Hershko
If such latency worries you, use in-memory structures to pull the original documents based on the IDs of the results. Your dataset is small enough to do that, really. But I suspect this is premature optimization that you are trying to do. -- Itamar Syn-Hershko http://code972.com | @synhershko