bq: The index size is only 1 M records. A 10 times of the record size (> 10M) will likely bring the total response time to > 1 second
This is an extrapolation you simply cannot make. Plus you cannot really tell anything from just a few queries about system performance. In fact you must disregard the first few queries due to loading Lucene indexes into memory. Plus you cannot extrapolate from just a few queries. Part of the time is loading the low-level Lucene caches for querying. And I'm assuming that time times you're reportinga re QTimes, but if they're not then there's the time spent assembling the response packet (i.e. reading/decompressing the data to get the stored data) which is almost entirely independent of the number of docs. In short, I don't have faith that your test methodology is reliable (although kudos for having methodology at all, lots of people don't!). And I'm 99.99% sure that you can't rely on the calculation that 10X the number of docs is 10X the response time. Best, Erick On Tue, Jun 30, 2015 at 2:51 PM, wwang525 <wwang...@gmail.com> wrote: > Hi All, > > I did many tests with very consistent test results. Each query was executed > after re-indexing, and only one request was sent to query the index. I > disabled filterCache and queryResultCache for this test based on Erick's > recommendation. > > The test document was posted to this email list earlier. Briefly, the query > without grouping and faceting took about 60 ms, and grouping on top of the > same query adds about 15 ms. However, the faceting adds additional 70 ms, > brings it to 140 ms > > The index size is only 1 M records. A 10 times of the record size (> 10M) > will likely bring the total response time to > 1 second for these two > queries. My goal is to make the query as performant as possible so that we > can achieve a < 1 second response time under load. > > Is a 50 ms to 60 ms response time (single request scenario) a bit too long > for 1M records with Solr? Is the faceting taking too long (70 ms)to > process? > > Thanks > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/How-to-do-a-Data-sharding-for-data-in-a-database-table-tp4212765p4215019.html > Sent from the Solr - User mailing list archive at Nabble.com.
