Thank you, these are useful tips. We were previously working with a 4GB heap and getting OOMs in Solr while updating (probably from the analysers) that would cause the index writer to close with what’s called a “tragic” error in the writer code. Only a hard restart of the service could bring it back. There are certain documents that function like poison and trigger this error every time. Haven’t had time to isolate and create a test case, so throwing RAM at it is a stopgap.
When I do, I’ll file an issue. > On 2 Mar 2017, at 18:28, Walter Underwood <wun...@wunderwood.org> wrote: > > 6.4.0 added a lot of metrics to low-level calls. That makes many operations > slow. Go back to 6.3.0 or wait for 6.4.2. > > Meanwhile, stop running optimize. You almost certainly don’t need it. > > 24 GB is a huge heap. Do you really need that? We run a 15 million doc index > with an 8 GB heap (Java 8u121, G1 collector). I recommend a smaller heap so > the OS can use that RAM to cache file buffers. > > wunder > Walter Underwood > wun...@wunderwood.org > http://observer.wunderwood.org/ (my blog) > > >> On Mar 2, 2017, at 7:04 AM, Caruana, Matthew <mcaru...@icij.org> wrote: >> >> I’m currently performing an optimise operation on a ~190GB index with about >> 4 million documents. The process has been running for hours. >> >> This is surprising, because the machine is an EC2 r4.xlarge with four cores >> and 30GB of RAM, 24GB of which is allocated to the JVM. >> >> The load average has been steady at about 1.3. Memory usage is 25% or less >> the whole time. iostat reports ~6% util. >> >> What gives? >> >> Running Solr 6.4.1. >
