How much of your memory are you allocating to the JVM and how much are you leaving free?
If you don't leave enough free memory for the OS, the OS won't have a large enough disk cache, and you will be hitting the disk for lots of queries. You might want to monitor your Disk I/O using iostat and look at the iowait. If you are doing phrase queries and your *prx file is significantly larger than the available memory then when a slow phrase query hits Solr, the contention for disk I/O with other queries could be slowing everything down. You might also want to look at the 90th and 99th percentile query times in addition to the average. For our large indexes, we found at least an order of magnitude difference between the average and 99th percentile queries. Again, if Solr gets hit with a few of those 99th percentile slow queries and your not hitting your caches, chances are you will see serious contention for disk I/O.. Of course if you don't see any waiting on i/o, then your bottleneck is probably somewhere else:) See http://www.hathitrust.org/blogs/large-scale-search/slow-queries-and-common-words-part-1 for more background on our experience. Tom Burton-West University of Michigan Library www.hathitrust.org > > On Thu, Mar 11, 2010 at 9:39 AM, Siddhant Goel <siddhantg...@gmail.com > >wrote: > > > Hi everyone, > > > > I have an index corresponding to ~2.5 million documents. The index size > is > > 43GB. The configuration of the machine which is running Solr is - Dual > > Processor Quad Core Xeon 5430 - 2.66GHz (Harpertown) - 2 x 12MB cache, > 8GB > > RAM, and 250 GB HDD. > > > > I'm observing a strange trend in the queries that I send to Solr. The > query > > times for queries that I send earlier is much lesser than the queries I > > send > > afterwards. For instance, if I write a script to query solr 5000 times > > (with > > 5000 distinct queries, most of them containing not more than 3-5 words) > > with > > 10 threads running in parallel, the average times for queries goes from > > ~50ms in the beginning to ~6000ms. Is this expected or is there > something > > wrong with my configuration. Currently I've configured the > queryResultCache > > and the documentCache to contain 2048 entries (hit ratios for both is > close > > to 50%). > > > > Apart from this, a general question that I want to ask is that is such a > > hardware enough for this scenario? I'm aiming at achieving around 20 > > queries > > per second with the hardware mentioned above. > > > > Thanks, > > > > Regards, > > > > -- > > - Siddhant > > > -- - Siddhant -- View this message in context: http://old.nabble.com/Solr-Performance-Issues-tp27864278p27868456.html Sent from the Solr - User mailing list archive at Nabble.com.
