It'd help if you could gist/pastebin/etc a query example. Also your current ES and java need updating, there are known issues with java <1.7u55, and you will always see performance boosts running the latest version of ES.
That aside, what is your current resource utilisation like? Are you seeing lots of cache evictions, high heap use, high CPU, IO delays? On 13 February 2015 at 07:32, Jay Danielian <jay.daniel...@circleback.com> wrote: > I know this is difficult to answer, the real answer is always "It Depends" > :) But I am going to go ahead and hope I get some feedback here. > > We are mainly using ES to issue terms searches against fields that are > non-analyzed. We are using ES like a key value store, where once the match > is found we parse the _source JSON and return our model. We are doing > contact lookups, searching against (last_name AND (phone_number OR email)). > We are issuing constant_score queries with term filters for the terms > mentioned above. No aggregations, no sorting, no scripts, etc. Using > JMeter, we were maxing out at around 500 search requests / sec. Average > request time was taking around 7 seconds to complete. When the test would > fire up, the ThreadPool Search Queue would spike to 1000 on each node and > CPU would be maxed out, then once it finished everything would return to > normal. So it appears healthy, and we wouldn't get any errors - just > nowhere close to the performance we are looking for. > > Setup details > - Index size 100GB with two different document mappings in the index. > Roughly 500M documents > - three nodes c3.4xl instances on EC2 using pIOPS SSD EBS disks (although > NOT RAID 0 - just one big volume) > - each server node on EC2 has 30GB RAM, 16GB on heap, rest for OS > - we have set mlockall on our instances > - 3 nodes are split into 6 shards for the main index > - Index is read only after it is loaded - we don't update the index ever, > it is only for querying > - ES version 1.3.3 Java 1.7.0_51 > - each server has 16 cores / node and 48 search threads with queue length > of 1000 > > Assuming no stemming, free text queries - just term matching, how can we > increase the throughput and decrease the response time for the ES queries? > is 500 requests / sec at the top end? > Do we just need many more servers if we really want 3000 requests / sec ? > I have read that scaling out is better for ES vs scaling up. But it feels > that the current server farm should deliver better performance. > > Any help or tuning advice would be really appreciated. We have looked at > many slideshares, blog posts from found.no, elasticseearch.org, etc - and > can't really pinpoint a way to improve our setup. > > Thanks! > > JD > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elasticsearch+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/47b93b84-d929-4cad-becd-31581cd4c574%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/47b93b84-d929-4cad-becd-31581cd4c574%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_JtmLUJg7W3bc_7t%3Dti9Bb%2B7YFOnRzQQ6cbNAEKh2SMw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
