Thank you, Yonik! I see the Jira issue you created and am guessing it's due to this issue. We're going to remove replicateAfter="startup" in the mean-time to see if that helps (assuming this is the issue the jira ticket described).
I appreciate you taking a look at this. Thanks -Trey On Fri, Apr 15, 2011 at 2:58 PM, Yonik Seeley <yo...@lucidimagination.com>wrote: > I can reproduce this with the example server w/ your deletionPolicy > and replicationHandler configs. > I'll dig further to see what's behind this behavior. > > -Yonik > http://www.lucenerevolution.org -- Lucene/Solr User Conference, May > 25-26, San Francisco > > On Fri, Apr 15, 2011 at 1:14 PM, Trey Grainger <solrt...@gmail.com> wrote: > > I was just hoping someone might be able to point me in the right > direction > > here. We just upgraded from Solr 1.4 to Solr 3.1 this past week and > we're > > having issues running out of disk space on our Master servers. Our > Master > > has dozens of cores. We have a script that kicks off once per day to do > a > > rolling optimize. The script optimizes a single core, waits 5 minutes to > > give the server some breathing room to catch up on indexing in a non-i/o > > intensive state, and then moves onto the next core (repeating until > done). > > > > The problem we are facing is that under Solr 1.4, the old index files > were > > deleted very quickly after each optimize, but under Solr 3.1, the old > index > > files hang around for hours... in many cases they don't disappear until > we > > restart Solr completely. This is leading to us running out of disk > space, > > as each core's index doubles in size during the optimize process and > stays > > that way until the next solr restart. > > > > I was just wondering if anyone could point me to some specific changes or > > settings which may be leading to the difference between solr versions (or > > any other environmental issues you may know about). I see several > tickets > > in Jira about similar issues, but they mostly appear to have been > resolved > > in the past. > > > > Has anyone else seen this behavior under Solr 3.1, or do you think we may > be > > missing some kind of new configuration setting? > > > > For reference, we are running on 64bit RedHat Linux. This is what I have > > right now: [From SolrConfig.xml]: > > <reopenReaders>true</reopenReaders> > > > > <requestHandler name="/replication" class="solr.ReplicationHandler"> > > <lst name="master"> > > <str name="replicateAfter">commit</str> > > <str name="replicateAfter">optimize</str> > > <str name="replicateAfter">startup</str> > > </lst> > > </requestHandler> > > > > <updateHandler class="solr.DirectUpdateHandler2"> > > <autoCommit> > > <maxDocs>100000</maxDocs> > > <maxTime>300000</maxTime> > > </autoCommit> > > </updateHandler> > > > > <deletionPolicy class="solr.SolrDeletionPolicy"> > > <str name="keepOptimizedOnly">false</str> > > <str name="maxCommitsToKeep">1</str> > > </deletionPolicy> > > > > > > Thanks in advance, > > > > -Trey > > >
