Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-25 Thread Jaco
Hi Noble, Great stuff, no problem, I really think the Solr development team is excellent and takes pride in delivering high quality software! And we're going into production with a brand new Solr based system in a few weeks as well, so I'm really happy that this is fixed now. Bye, Jaco. 2009/1

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-24 Thread Noble Paul നോബിള്‍ नोब्ळ्
hi Jaco, We owe you a bing THANK YOU. We were planning to roll out this feature into production in the next week or so. Our internal testing could not find this out. --Noble On Fri, Jan 23, 2009 at 6:36 PM, Jaco wrote: > Hi, > > I have tested this as well, looking fine! Both issues are inde

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Jaco
Hi, I have tested this as well, looking fine! Both issues are indeed fixed, and the index directory of the slaves gets cleaned up nicely. I will apply the changes to all systems I've got running and report back in this thread in case any issues are found. Thanks for the very fast help! I usually

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
I have opened an issue to track this https://issues.apache.org/jira/browse/SOLR-978 On Fri, Jan 23, 2009 at 5:22 PM, Noble Paul നോബിള്‍ नोब्ळ् wrote: > I tested with the patch > it has solved both the issues > > On Fri, Jan 23, 2009 at 5:00 PM, Shalin Shekhar Mangar > wrote: >> >> >> On Fri, Ja

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Noble Paul നോബിള്‍ नोब्ळ्
I tested with the patch it has solved both the issues On Fri, Jan 23, 2009 at 5:00 PM, Shalin Shekhar Mangar wrote: > > > On Fri, Jan 23, 2009 at 2:12 PM, Jaco wrote: >> >> Hi, >> >> I applied the patch and did some more tests - also adding some LOG.info() >> calls in delTree to see if it actual

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Shalin Shekhar Mangar
On Fri, Jan 23, 2009 at 2:12 PM, Jaco wrote: > Hi, > > I applied the patch and did some more tests - also adding some LOG.info() > calls in delTree to see if it actually gets invoked (LOG.info("START: > delTree: "+dir.getName()); at the start of that method). I don't see any > entries of this sho

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-23 Thread Jaco
Hi, I applied the patch and did some more tests - also adding some LOG.info() calls in delTree to see if it actually gets invoked (LOG.info("START: delTree: "+dir.getName()); at the start of that method). I don't see any entries of this showing up in the log file at all, so it looks like delTree d

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Shalin Shekhar Mangar
On Fri, Jan 23, 2009 at 12:15 AM, Noble Paul നോബിള്‍ नोब्ळ् < noble.p...@gmail.com> wrote: > I have attached a patch which logs the names of the files which could > not get deleted (which may help us diagnose the problem). If you are > comfortable applying a patch you may try it out. > I've commi

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
I am not sure if it was completely fixed. (This was related to a Lucene bug) But you can try w/ a recent build and confirm it for us. I have never encountered these during our tests in windows XP/Linux I have attached a patch which logs the names of the files which could not get deleted (which may

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Jeff Newburn
Few weeks ago is our version. Does this contribute to the directory issues and extra files that are left? On 1/22/09 10:33 AM, "Noble Paul നോബിള്‍ नोब्ळ्" wrote: > This was reported by another user and was fixed recently.Are you using > a recent version? > --Noble > > On Fri, Jan 23, 2009 at

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
This was reported by another user and was fixed recently.Are you using a recent version? --Noble On Fri, Jan 23, 2009 at 12:00 AM, Jeff Newburn wrote: > We have both. A majority of them are just empty but others have almost a > full index worth of files. I have also noticed that during a length

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Jeff Newburn
We have both. A majority of them are just empty but others have almost a full index worth of files. I have also noticed that during a lengthy index update the system will throw errors about how it cannot move one of the index files. Essentially on reindex the system does not replicate until an o

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Noble Paul നോബിള്‍ नोब्ळ्
Jeff , Do you see both the empty index. dirs as well as the extra files in the index? --Noble On Thu, Jan 22, 2009 at 10:37 PM, Jeff Newburn wrote: > We are seeing something very similar. Ours is intermittent and usually > happens a great deal on random days. Often it seems to occur during l

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Jeff Newburn
My apologies. No we are using linux, tomcat setup. On 1/22/09 9:15 AM, "Shalin Shekhar Mangar" wrote: > On Thu, Jan 22, 2009 at 10:37 PM, Jeff Newburn wrote: > >> We are seeing something very similar. Ours is intermittent and usually >> happens a great deal on random days. Often it seems to

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Shalin Shekhar Mangar
On Thu, Jan 22, 2009 at 10:37 PM, Jeff Newburn wrote: > We are seeing something very similar. Ours is intermittent and usually > happens a great deal on random days. Often it seems to occur during large > index updates on the master. > Jeff, is this also on a Windows box? -- Regards, Shalin S

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Jeff Newburn
We are seeing something very similar. Ours is intermittent and usually happens a great deal on random days. Often it seems to occur during large index updates on the master. On 1/22/09 8:58 AM, "Shalin Shekhar Mangar" wrote: > On Thu, Jan 22, 2009 at 10:18 PM, Jaco wrote: > >> Hm, I don't kn

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Shalin Shekhar Mangar
On Thu, Jan 22, 2009 at 10:18 PM, Jaco wrote: > Hm, I don't know what to do anymore. I tried this: > - Run Tomcat service as local administrator to overcome any permissioning > issues > - Installed latest nightly build (I noticed that item I mentioned before ( > http://markmail.org/message/yq2ram

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-22 Thread Jaco
Hm, I don't know what to do anymore. I tried this: - Run Tomcat service as local administrator to overcome any permissioning issues - Installed latest nightly build (I noticed that item I mentioned before ( http://markmail.org/message/yq2ram4f3jblermd) had been committed which is good - Build a sma

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
On Wed, Jan 21, 2009 at 3:42 PM, Jaco wrote: > Thanks for the fast replies! > > It appears that I made a (probably classical) error... I didnt' make the > change to solrconfig.xml to include the when applying the > upgrade. I include this now, but the slave is not cleaning up. Will this be > done

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Jaco
Thanks for the fast replies! It appears that I made a (probably classical) error... I didnt' make the change to solrconfig.xml to include the when applying the upgrade. I include this now, but the slave is not cleaning up. Will this be done at some point automatically? Can I trigger this? User a

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Shalin Shekhar Mangar
Hi, There shouldn't be so many files on the slave. Since the empty index.x folders are not getting deleted, is it possible that Solr process user does not enough privileges to delete files/folders? Also, have you made any changes to the IndexDeletionPolicy configuration? On Wed, Jan 21, 2009

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Noble Paul നോബിള്‍ नोब्ळ्
the index.xxx directories are supposed to be deleted (automatically). you can safely delete them. But, I am wondering why the index files in the slave did not get deleted. By default the deletionPolicy is KeepOnlyLastCommit. On Wed, Jan 21, 2009 at 2:15 PM, Jaco wrote: > Hi, > > I'm running So

Re: Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Rafał Kuć
Hello, > Hi, > I'm running Solr nightly build of 20.12.2008, with patch as discussed on > http://markmail.org/message/yq2ram4f3jblermd, using Solr replication. > On various systems running, I see that the disk space consumed on the slave > is much higher than on the master. One example: > - Mast

Solr Replication: disk space consumed on slave much higher than on master

2009-01-21 Thread Jaco
Hi, I'm running Solr nightly build of 20.12.2008, with patch as discussed on http://markmail.org/message/yq2ram4f3jblermd, using Solr replication. On various systems running, I see that the disk space consumed on the slave is much higher than on the master. One example: - Master: 30 GB in 138 fil