I was just hoping someone might be able to point me in the right direction
here. We just upgraded from Solr 1.4 to Solr 3.1 this past week and we're
having issues running out of disk space on our Master servers. Our Master
has dozens of cores. We have a script that kicks off once per day to do a
rolling optimize. The script optimizes a single core, waits 5 minutes to
give the server some breathing room to catch up on indexing in a non-i/o
intensive state, and then moves onto the next core (repeating until done).
The problem we are facing is that under Solr 1.4, the old index files were
deleted very quickly after each optimize, but under Solr 3.1, the old index
files hang around for hours... in many cases they don't disappear until we
restart Solr completely. This is leading to us running out of disk space,
as each core's index doubles in size during the optimize process and stays
that way until the next solr restart.
I was just wondering if anyone could point me to some specific changes or
settings which may be leading to the difference between solr versions (or
any other environmental issues you may know about). I see several tickets
in Jira about similar issues, but they mostly appear to have been resolved
in the past.
Has anyone else seen this behavior under Solr 3.1, or do you think we may be
missing some kind of new configuration setting?
For reference, we are running on 64bit RedHat Linux. This is what I have
right now: [From SolrConfig.xml]:
<reopenReaders>true</reopenReaders>
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="master">
<str name="replicateAfter">commit</str>
<str name="replicateAfter">optimize</str>
<str name="replicateAfter">startup</str>
</lst>
</requestHandler>
<updateHandler class="solr.DirectUpdateHandler2">
<autoCommit>
<maxDocs>100000</maxDocs>
<maxTime>300000</maxTime>
</autoCommit>
</updateHandler>
<deletionPolicy class="solr.SolrDeletionPolicy">
<str name="keepOptimizedOnly">false</str>
<str name="maxCommitsToKeep">1</str>
</deletionPolicy>
Thanks in advance,
-Trey
