Hi I've inherited a Solr config and am doing some sanity checks before making some updates, I'm concerned about the memory settings.
System has 1 index in 2 shards split across 2 Ubuntu 64 bit nodes, each node has 32 CPU cores and 132GB RAM, we index around 500k files a day spread out over the day in batches every 10 minutes, a portion of these are updates to existing content, maybe 5-10%. Currently MergeFactor is set to 2 and commit settings are: <autoCommit> <maxTime>60000</maxTime> <openSearcher>false</openSearcher> </autoCommit> <autoSoftCommit> <maxTime>900000</maxTime> </autoSoftCommit> Currently each node has around 25M docs in with an index size of 45GB, we prune the data every few weeks so it never gets much above 35M docs per node. On reading I've seen a recommendation that we should be using MMapDirectory, currently it's set to NRTCachingDirectoryFactory. However currently the JVM is configured with -Xmx131072m, and for MMapDirectory I've read you should use less memory for the JVM so there is more available for the OS caching. Looking at the dashboard in the JVM memory usage I see: Not sure I understand the 3 bands, assume 127.81 is Max, dark grey is in use at the moment and the light grey is allocated as it was used previously but not been cleaned up yet? I'm trying to understand if this will help me know how much would be a good value to change Xmx to, i.e. say 64GB based on light grey? Additionally once I've changed the max heap size is it a simple case of changing the config to use MMapDirectory or are there things i need to watch out for? Thanks Si