How many replicas do you have configured for the index? Christian
On Thursday, February 12, 2015 at 8:32:28 PM UTC, Jay Danielian wrote: > > I know this is difficult to answer, the real answer is always "It Depends" > :) But I am going to go ahead and hope I get some feedback here. > > We are mainly using ES to issue terms searches against fields that are > non-analyzed. We are using ES like a key value store, where once the match > is found we parse the _source JSON and return our model. We are doing > contact lookups, searching against (last_name AND (phone_number OR email)). > We are issuing constant_score queries with term filters for the terms > mentioned above. No aggregations, no sorting, no scripts, etc. Using > JMeter, we were maxing out at around 500 search requests / sec. Average > request time was taking around 7 seconds to complete. When the test would > fire up, the ThreadPool Search Queue would spike to 1000 on each node and > CPU would be maxed out, then once it finished everything would return to > normal. So it appears healthy, and we wouldn't get any errors - just > nowhere close to the performance we are looking for. > > Setup details > - Index size 100GB with two different document mappings in the index. > Roughly 500M documents > - three nodes c3.4xl instances on EC2 using pIOPS SSD EBS disks (although > NOT RAID 0 - just one big volume) > - each server node on EC2 has 30GB RAM, 16GB on heap, rest for OS > - we have set mlockall on our instances > - 3 nodes are split into 6 shards for the main index > - Index is read only after it is loaded - we don't update the index ever, > it is only for querying > - ES version 1.3.3 Java 1.7.0_51 > - each server has 16 cores / node and 48 search threads with queue length > of 1000 > > Assuming no stemming, free text queries - just term matching, how can we > increase the throughput and decrease the response time for the ES queries? > is 500 requests / sec at the top end? > Do we just need many more servers if we really want 3000 requests / sec ? > I have read that scaling out is better for ES vs scaling up. But it feels > that the current server farm should deliver better performance. > > Any help or tuning advice would be really appreciated. We have looked at > many slideshares, blog posts from found.no, elasticseearch.org, etc - and > can't really pinpoint a way to improve our setup. > > Thanks! > > JD > > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fbca444d-34f1-447e-a55f-fb4218d571cc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
