Btw, confirmed that this doesn't happen on our development stage with 3.6.
On Wed, May 16, 2012 at 3:59 PM, Jasper Floor <jasper.fl...@m4n.nl> wrote: > The slave index does indeed grow over a period of time regardless of > restarts. We do run on 1.4 however. We will be updating to 3.6 very > soon however so I will see how that works out. Actually we should be > able to see this on our staging platform. > > thanks everyone. > > mvg, > Jasper > > On Mon, May 14, 2012 at 4:40 PM, Bill Bell <billnb...@gmail.com> wrote: >> This is a known issue in 1.4 especially in Windows. Some of it was resolved >> in 3x. >> >> Bill Bell >> Sent from mobile >> >> >> On May 14, 2012, at 5:54 AM, Erick Erickson <erickerick...@gmail.com> wrote: >> >>> Hmmm, replication will require up to twice the space of the >>> index _temporarily_, just checking if that's what you're seeing.... >>> But that should go away reasonably soon. Out of curiosity, what >>> happens if you restart your server, do the extra files go away? >>> >>> But it sounds like your index is growing over a longer period of time >>> than just a single replication, is that true? >>> >>> Best >>> Erick >>> >>> On Fri, May 11, 2012 at 6:03 AM, Jasper Floor <jasper.fl...@m4n.nl> wrote: >>>> Hi, >>>> >>>> On Thu, May 10, 2012 at 5:59 PM, Otis Gospodnetic >>>> <otis_gospodne...@yahoo.com> wrote: >>>>> Hi Jasper, >>>> >>>> Sorry, I should've added more technical info wihtout being prompted. >>>> >>>>> Solr does handle that for you. Some more stuff to share: >>>>> >>>>> * Solr version? >>>> >>>> 1.4 >>>> >>>>> * JVM version? >>>> 1.7 update 2 >>>> >>>>> * OS? >>>> Debian (2.6.32-5-xen-amd64) >>>> >>>>> * Java replication? >>>> yes >>>> >>>>> * Errors in Solr logs? >>>> no >>>> >>>>> * deletion policy section in solrconfig.xml? >>>> missing I would say, but I don't see this on the replication wiki page. >>>> >>>> This is what we have configured for replication: >>>> >>>> <requestHandler name="/replication" class="solr.ReplicationHandler" > >>>> <lst name="slave"> >>>> >>>> <str >>>> name="masterUrl">${solr.master.url}/df-stream-store/replication</str> >>>> >>>> <str name="pollInterval">00:20:00</str> >>>> <str name="compression">internal</str> >>>> <str name="httpConnTimeout">5000</str> >>>> <str name="httpReadTimeout">10000</str> >>>> >>>> </lst> >>>> </requestHandler> >>>> >>>> We will be updating to 3.6 fairly soon however. To be honest, from >>>> what I've read, the Solr cloud is what we really want in the future >>>> but we will have to be patient for that. >>>> >>>> thanks in advance >>>> >>>> mvg, >>>> Jasper >>>> >>>>> You may also want to look at your Index report in SPM >>>>> (http://sematext.com/spm) before/during/after replication and share what >>>>> you see. >>>>> >>>>> Otis >>>>> ---- >>>>> Performance Monitoring for Solr / ElasticSearch / HBase - >>>>> http://sematext.com/spm >>>>> >>>>> >>>>> >>>>> ----- Original Message ----- >>>>>> From: Jasper Floor <jasper.fl...@m4n.nl> >>>>>> To: solr-user@lucene.apache.org >>>>>> Cc: >>>>>> Sent: Thursday, May 10, 2012 9:08 AM >>>>>> Subject: slave index not cleaned >>>>>> >>>>>> Perhaps I am missing the obvious but our slaves tend to run out of >>>>>> disk space. The index sizes grow to multiple times the size of the >>>>>> master. So I just toss all the data and trigger a replication. >>>>>> However, can't solr handle this for me? >>>>>> >>>>>> I'm sorry if I've missed a simple setting which does this for me, but >>>>>> if its there then I have missed it. >>>>>> >>>>>> mvg >>>>>> Jasper >>>>>>