I can reproduce this with the example server w/ your deletionPolicy and replicationHandler configs. I'll dig further to see what's behind this behavior.
-Yonik http://www.lucenerevolution.org -- Lucene/Solr User Conference, May 25-26, San Francisco On Fri, Apr 15, 2011 at 1:14 PM, Trey Grainger <solrt...@gmail.com> wrote: > I was just hoping someone might be able to point me in the right direction > here. We just upgraded from Solr 1.4 to Solr 3.1 this past week and we're > having issues running out of disk space on our Master servers. Our Master > has dozens of cores. We have a script that kicks off once per day to do a > rolling optimize. The script optimizes a single core, waits 5 minutes to > give the server some breathing room to catch up on indexing in a non-i/o > intensive state, and then moves onto the next core (repeating until done). > > The problem we are facing is that under Solr 1.4, the old index files were > deleted very quickly after each optimize, but under Solr 3.1, the old index > files hang around for hours... in many cases they don't disappear until we > restart Solr completely. This is leading to us running out of disk space, > as each core's index doubles in size during the optimize process and stays > that way until the next solr restart. > > I was just wondering if anyone could point me to some specific changes or > settings which may be leading to the difference between solr versions (or > any other environmental issues you may know about). I see several tickets > in Jira about similar issues, but they mostly appear to have been resolved > in the past. > > Has anyone else seen this behavior under Solr 3.1, or do you think we may be > missing some kind of new configuration setting? > > For reference, we are running on 64bit RedHat Linux. This is what I have > right now: [From SolrConfig.xml]: > <reopenReaders>true</reopenReaders> > > <requestHandler name="/replication" class="solr.ReplicationHandler"> > <lst name="master"> > <str name="replicateAfter">commit</str> > <str name="replicateAfter">optimize</str> > <str name="replicateAfter">startup</str> > </lst> > </requestHandler> > > <updateHandler class="solr.DirectUpdateHandler2"> > <autoCommit> > <maxDocs>100000</maxDocs> > <maxTime>300000</maxTime> > </autoCommit> > </updateHandler> > > <deletionPolicy class="solr.SolrDeletionPolicy"> > <str name="keepOptimizedOnly">false</str> > <str name="maxCommitsToKeep">1</str> > </deletionPolicy> > > > Thanks in advance, > > -Trey >
