Hi guys, We're on solr 3.6.1 and I've read the discussions about whether to optimize or not to optimize. I decided to try not optimizing our index as was recommended. We have a little over 15 million docs in our biggest index and a 32gb heap for our jvm. So without the optimizes the index folder seemed to grow in size and quantity of files. There seemed to be an upper limit but eventually it hit 300 files consuming 26gb of space and that seemed to push our slave farm over the edge and we started getting the dreaded OOMs. We have continuous indexing activity, so I stopped the indexer and manually ran an optimize which made the index become 9 files consuming 15gb of space and our slave farm started having acceptable memory usage. Our merge factor is 10, we're on java 7. Before optimizing, I tried on one slave machine to go with the latest JVM and tried switching from the CMS GC to the G1GC but it hit OOM condition even faster. So it seems like I have to continue to schedule a regular optimize. Right now it has been a couple of days since running the optimize and the index is slowly growing bigger, now up to a bit over 19gb. What do you guys think? Did I miss something that would make us able to run without doing an optimize?
Robert (Robi) Petersen Senior Software Engineer Search Department
