Index files not being deleted

[AgentHubcap]

I've got an issue with my index files not being deleted.  I have a cron job
set up to kick off the index process every 5 minutes (test box, with little
data).  The process is as follows:

- Delete all index files via a delete command
- Loop through all the records in the database and index them
- Commits are triggered every 250 records
- An optimize is done at the end (twice because I read there was an issue
with the optimize command)

What is happening is that the number of open files keeps growing every time
the cron job runs, and the disk space is eventually used up by deleted files
(something isn't closing the handle on the file, and thus the file is never
really deleted)

My question is what exactly do I need to do to close these files?  This is
for an index only -- no searching is taking place.  I'm also using the
embedded version if that makes a difference...
-- 
View this message in context: 
http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12869275
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Index files not being deleted

[Ryan McKinley]

- Delete all index files via a delete command


make sure to optimize after deleting the docs -- optimize has lucene get 
rid of deleted files rather then appending them to the end of the index.


what version of solr are you running?  if you are running 1.3-dev 
deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving 
there just for that though


ryan

Re: Index files not being deleted

[AgentHubcap]

I'm running 1.2.

Acutally, i am doing an optimize after I delete the indexes.  (twice, as I
read there was an issue with the optimize).  Do I need to close something
manually?

Here's my optimize code:

private void optimize() throws IOException
{
UpdateHandler updateHandler = 
SolrCore.getSolrCore().getUpdateHandler();
CommitUpdateCommand commitcmd = new CommitUpdateCommand(false);
commitcmd.optimize = true;
updateHandler.commit(commitcmd);
updateHandler.close();
}




ryantxu wrote:
> 
>> 
>> - Delete all index files via a delete command
> 
> make sure to optimize after deleting the docs -- optimize has lucene get 
> rid of deleted files rather then appending them to the end of the index.
> 
> what version of solr are you running?  if you are running 1.3-dev 
> deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving 
> there just for that though
> 
> ryan
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12870871
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Index files not being deleted

[AgentHubcap]

As it turns out I was modifying code that wasn't being run.  Running an
optimize after deleting did solve my problem.  =)



AgentHubcap wrote:
> 
> I'm running 1.2.
> 
> Acutally, i am doing an optimize after I delete the indexes.  (twice, as I
> read there was an issue with the optimize).  Do I need to close something
> manually?
> 
> Here's my optimize code:
> 
>   private void optimize() throws IOException
>   {
>   UpdateHandler updateHandler = 
> SolrCore.getSolrCore().getUpdateHandler();
>   CommitUpdateCommand commitcmd = new CommitUpdateCommand(false);
>   commitcmd.optimize = true;
>   updateHandler.commit(commitcmd);
>   updateHandler.close();
>   }
> 
> 
> 
> 
> ryantxu wrote:
>> 
>>> 
>>> - Delete all index files via a delete command
>> 
>> make sure to optimize after deleting the docs -- optimize has lucene get 
>> rid of deleted files rather then appending them to the end of the index.
>> 
>> what version of solr are you running?  if you are running 1.3-dev 
>> deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving 
>> there just for that though
>> 
>> ryan
>> 
>> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a12885332
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Index files not being deleted

[AgentHubcap]

So, this problem came up again.  Now it only happens in a linux environment
when searches are being conducted while an index is running.

Does anything need to be closed on the searching side?



AgentHubcap wrote:
> 
> As it turns out I was modifying code that wasn't being run.  Running an
> optimize after deleting did solve my problem.  =)
> 
> 
> 
> AgentHubcap wrote:
>> 
>> I'm running 1.2.
>> 
>> Acutally, i am doing an optimize after I delete the indexes.  (twice, as
>> I read there was an issue with the optimize).  Do I need to close
>> something manually?
>> 
>> Here's my optimize code:
>> 
>>  private void optimize() throws IOException
>>  {
>>  UpdateHandler updateHandler =
>> SolrCore.getSolrCore().getUpdateHandler();
>>  CommitUpdateCommand commitcmd = new CommitUpdateCommand(false);
>>  commitcmd.optimize = true;
>>  updateHandler.commit(commitcmd);
>>  updateHandler.close();
>>  }
>> 
>> 
>> 
>> 
>> ryantxu wrote:
>>> 
>>>> 
>>>> - Delete all index files via a delete command
>>> 
>>> make sure to optimize after deleting the docs -- optimize has lucene get 
>>> rid of deleted files rather then appending them to the end of the index.
>>> 
>>> what version of solr are you running?  if you are running 1.3-dev 
>>> deleting *:* is fast -- if you aren't using 1.3, i don't suggest moving 
>>> there just for that though
>>> 
>>> ryan
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: 
http://www.nabble.com/Index-files-not-being-deleted-tf4512068.html#a13128043
Sent from the Solr - User mailing list archive at Nabble.com.