Thank you for your response. Our index has about 25,000,000 docs. Is this quite difficult to achieve 1000 qps with single Solr instance even if all index files are cached on OS page cache?
And as you pointed out, there are lots of minor GC during the test, I will try to resize NewRatio. Thanks, Yasufumi. 2019年9月30日(月) 22:18 Shawn Heisey <apa...@elyograg.org>: > On 9/29/2019 11:44 PM, Yasufumi Mizoguchi wrote: > > I am trying some tests to confirm if single Solr instance can perform > over > > 1000 queries per second(!). > > In general, I would never expect a single instance to handle a large > number of queries per second unless the index is REALLY small -- dozens > or hundreds of very small documents. A 60GB index definitely does not > qualify. > > I don't think it will be possible to handle 1000 queries per second with > a single server even on a really small index, but I've never actually > tried. > > > But now, although CPU usage is 40% or so and iowait is almost 0%, > > throughput does not increase over 60 queries per second. > > A query rate of 60 per second is pretty good with an index size of 60GB. > The low iowait would tend to confirm that the index is well cached by > the OS. > > If you need to handle 1000 queries per second, you need more copies of > your index on additional Solr servers, with something in the mix to > perform load balancing. > > Some thoughts: > > With your -XX:MaxNewSize=128m setting, you are likely causing garbage > collection to occur VERY frequently, which will slow things down. > Solr's default GC settings include -XX:NewRatio=3 so that the new > generation will be much larger than what you have set. A program like > Solr that allocates a lot of memory will need a fairly large new > generation. > > I agree with the idea of setting the heap to 31GB. Setting it to 31GB > will actually leave more memory available to Solr than setting it to > 32GB, because of the decreased pointer sizes. > > Definitely check what Erick mentioned. If you're seeing what he > described, adjusting how threads work might get you more throughput. > But look into your new generation sizing first. > > Thanks, > Shawn >
