Hi, I used to think this, too, but have learned this not to be entirely true. We had a customer with a query rate of a few hundred QPS and 32 or 64 GB RAM (don't recall which any more) and a pretty large JVM heap. Most queries were very fast, but once in a while a query would be very slow. GC, we thought! So the initial thinking was was - must be that big heap of theirs. But.... long story short, instead of making the heap smaller we just tuned the JVM and took care of those slow queries. Using SPM (link in sig) and seeing GC info (collection counts, times, heap size, etc.) was invaluable!
Otis ---- Performance Monitoring SaaS for Solr - http://sematext.com/spm/solr-performance-monitoring/index.html - FREE! >________________________________ > From: Robert Stewart <bstewart...@gmail.com> >To: solr-user@lucene.apache.org >Sent: Thursday, December 15, 2011 2:16 PM >Subject: Re: Core overhead > >One other thing I did not mention is GC pauses. If you have smaller >heap sizes, you would have less very long GC pauses, so that can be an >advantage having many cores (if cores are distributed into seperate >SOLR instances, as seperate processes). I think you can expect 1 >second pause for each GB of heap size in worst case. > > > >On Thu, Dec 15, 2011 at 2:14 PM, Robert Stewart <bstewart...@gmail.com> wrote: >> It is true number of terms may be much more than N/10 (or even N for >> each core), but it is the number of docs per term that will really >> matter. So you can have N terms in each core but each term has 1/10 >> number of docs on avg. >> >> >> >> >> 2011/12/15 Yury Kats <yuryk...@yahoo.com>: >>> On 12/15/2011 1:07 PM, Robert Stewart wrote: >>> >>>> I think overall memory usage would be close to the same. >>> >>> Is this really so? I suspect that the consumed memory is in direct >>> proportion to the number of terms in the index. I also suspect that >>> if I divided 1 core with N terms into 10 smaller cores, each smaller >>> core would have much more than N/10 terms. Let's say I'm indexing >>> English texts, it's likely that all smaller cores would have almost >>> the same number of terms, close to the original N. Not so? > > >
