On 12/13/2010 9:15 PM, Mark wrote:
No cache warming queries and our machines have 8g of memory in them with about 5120m of ram dedicated to so Solr. When our index is around 10-11g in size everything runs smoothly. At around 20g+ it just falls apart.
I just replied to your new email thread, called "Need some guidance on solr-config settings." Then I saw this.
I'd recommend two things in addition to the autoWarmCount reductions I mentioned in the other thread. The first is shrink your Solr memory footprint. Reduce the size of your caches and your max heap size so you have more RAM available for the operating system disk cache. If you find that you can't reduce your max heap size very much because you are using memory-hungry Solr features like complex faceting, we come to the second recommendation, which is to install more memory. 16GB would be a good start, but the more you can get, the better. Referring back to my Solr installation, each shard has 9GB of RAM, with a max heap size of only 2GB. This leaves 7GB for the disk cache, slightly less than half the index size. At this time, I am not doing anything real complex with Solr, but all of the faceting tests that I have done work with no memory trouble.
Anything that you can do to decrease the index size is also a good idea. Only store enough information in your index to populate the initial search results screen. When your users open a particular search result, have your application grab all the detail information from the original data source, not Solr.
