Hi,
I'm pretty sure this has been asked before, but I couldn't find a
complete answer in the forum archive. Here are my questions,
1) When solr starts up what does it loads up in the memory? Let's say
I've 4 cores with each core 50G in size. When Solr comes up how much
of it would be loaded in memory?
2) How much memory is required during index time? If I'm committing
50K records at a time (1 record = 1KB) using solrj, how much memory do
I need to give to Solr.
3) Is there a minimum memory requirement by Solr to maintain a certain
size index? Is there any benchmark on this?
Here are some of my configuration from solrconfig.xml,
1) <ramBufferSizeMB>64</ramBufferSizeMB>
2) All the caches (under query tag) are commented out
3) Few others,
a) <enableLazyFieldLoading>true</enableLazyFieldLoading> ==>
would this require memory?
b) <queryResultWindowSize>50</queryResultWindowSize>
c) <queryResultMaxDocsCached>200</queryResultMaxDocsCached>
d) <HashDocSet maxSize="3000" loadFactor="0.75"/>
e) <useColdSearcher>false</useColdSearcher>
f) <maxWarmingSearchers>2</maxWarmingSearchers>
The problem we are having is following,
I've given Solr RAM of 6G. As the total index size (all cores
combined) start growing the Solr memory consumption goes up. With 800
million documents, I see Solr already taking up all the memory at
startup. After that the commits, searches everything become slow. We
will be having distributed setup with multiple Solr instances (around
8) on four boxes, but our requirement is to have each Solr instance at
least maintain around 1.5 billion documents.
We are trying to see if we can somehow reduce the Solr memory
footprint. If someone can provide a pointer on what parameters affect
memory and what effects it has we can then decide whether we want that
parameter or not. I'm not sure if there is any minimum Solr
requirement for it to be able maintain large indexes. I've used Lucene
before and that didn't require anything by default - it used up memory
only during index and search times - not otherwise.
Any help is very much appreciated.
Thanks,
-vivek
