Index files not being deleted
I've got an issue with my index files not being deleted. I have a cron job set up to kick off the index process every 5 minutes (test box, with little data). The process is as follows: - Delete all index files via a delete command - Loop through all the records in the database and index them - Commits are triggered every 250 records - An optimize is done at the end (twice because I read there was an issue with the optimize command) What is happening is that the number of open files keeps growing every time the cron job runs, and the disk space is eventually used up by deleted files (something isn't closing the handle on the file, and thus the file is never really deleted) My question is what exactly do I need to do to close these files? This is for an index only -- no searching is taking place. I'm also using the embedded version if that makes a difference... -- View this message in context: http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12869275 Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index files not being deleted
- Delete all index files via a delete command make sure to optimize after deleting the docs -- optimize has lucene get rid of deleted files rather then appending them to the end of the index. what version of solr are you running? if you are running 1.3-dev deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving there just for that though ryan
Re: Index files not being deleted
I'm running 1.2. Acutally, i am doing an optimize after I delete the indexes. (twice, as I read there was an issue with the optimize). Do I need to close something manually? Here's my optimize code: private void optimize() throws IOException { UpdateHandler updateHandler = SolrCore.getSolrCore().getUpdateHandler(); CommitUpdateCommand commitcmd = new CommitUpdateCommand(false); commitcmd.optimize = true; updateHandler.commit(commitcmd); updateHandler.close(); } ryantxu wrote: > >> >> - Delete all index files via a delete command > > make sure to optimize after deleting the docs -- optimize has lucene get > rid of deleted files rather then appending them to the end of the index. > > what version of solr are you running? if you are running 1.3-dev > deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving > there just for that though > > ryan > > -- View this message in context: http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12870871 Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index files not being deleted
As it turns out I was modifying code that wasn't being run. Running an optimize after deleting did solve my problem. =) AgentHubcap wrote: > > I'm running 1.2. > > Acutally, i am doing an optimize after I delete the indexes. (twice, as I > read there was an issue with the optimize). Do I need to close something > manually? > > Here's my optimize code: > > private void optimize() throws IOException > { > UpdateHandler updateHandler = > SolrCore.getSolrCore().getUpdateHandler(); > CommitUpdateCommand commitcmd = new CommitUpdateCommand(false); > commitcmd.optimize = true; > updateHandler.commit(commitcmd); > updateHandler.close(); > } > > > > > ryantxu wrote: >> >>> >>> - Delete all index files via a delete command >> >> make sure to optimize after deleting the docs -- optimize has lucene get >> rid of deleted files rather then appending them to the end of the index. >> >> what version of solr are you running? if you are running 1.3-dev >> deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving >> there just for that though >> >> ryan >> >> > > -- View this message in context: http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12885332 Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index files not being deleted
So, this problem came up again. Now it only happens in a linux environment when searches are being conducted while an index is running. Does anything need to be closed on the searching side? AgentHubcap wrote: > > As it turns out I was modifying code that wasn't being run. Running an > optimize after deleting did solve my problem. =) > > > > AgentHubcap wrote: >> >> I'm running 1.2. >> >> Acutally, i am doing an optimize after I delete the indexes. (twice, as >> I read there was an issue with the optimize). Do I need to close >> something manually? >> >> Here's my optimize code: >> >> private void optimize() throws IOException >> { >> UpdateHandler updateHandler = >> SolrCore.getSolrCore().getUpdateHandler(); >> CommitUpdateCommand commitcmd = new CommitUpdateCommand(false); >> commitcmd.optimize = true; >> updateHandler.commit(commitcmd); >> updateHandler.close(); >> } >> >> >> >> >> ryantxu wrote: >>> >>>> >>>> - Delete all index files via a delete command >>> >>> make sure to optimize after deleting the docs -- optimize has lucene get >>> rid of deleted files rather then appending them to the end of the index. >>> >>> what version of solr are you running? if you are running 1.3-dev >>> deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving >>> there just for that though >>> >>> ryan >>> >>> >> >> > > -- View this message in context: http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a13128043 Sent from the Solr - User mailing list archive at Nabble.com.