Further to what Erick says, recent versions of lucene hang on to
unclosed readers longer than old versions used to.
lsof -p pid can be useful here: run it and grep for deleted files
still being held open.
--
Ian.
On Mon, Mar 14, 2011 at 6:13 PM, Erick Erickson <erickerick...@gmail.com> wrote:
> This sounds like you're not closing your index searchers and the file system
> is keeping them around. On the Unix box, does hour index space reappear
> just by restarting the process?
>
> Not using reopen correctly is sometimes the culprit, you need something like
> this (taken from the javadocs).
> IndexReader reader = ...
> ...
> IndexReader new = r.reopen();
> if (new != reader) {
> ... // reader was reopened
> reader.close();
> }
> reader = new;
>
> ******************
> the mistake is to write something like:
> reader = reader.reopen();
> in which case the underlying reader is never closed.
>
> Best
> Erick
>
> On Mon, Mar 14, 2011 at 1:55 PM, Sirish Vadala <sirishre...@gmail.com> wrote:
>> Hello All:
>>
>> Background:
>> I have a text based search engine implemented in Java using Lucene 3.0.
>> Indexing and re-indexing happens every night at 1 am as a scheduled process.
>> The index size is around 1 gig and is recreated every night.
>>
>> Issues
>> 1. Now I have a peculiar problem that happens only on my UNIX server. Every
>> night after deleting the existing indexes and recreating the new, the disk
>> loses around 1 gig space. When I look into the directory, I see a new file
>> created with same size as the previous one, still overall space is lost.
>>
>> 2. Also, there is an issue with RAM memory. During indexing the memory
>> occupancy is high, which is understandable. However, the memory occupancy
>> remains the same even after completing the indexing process and this keeps
>> increasing day by day until the server runs out of memory in a few weeks.
>> This happens both on my Windows and Unix servers.
>>
>> Any help or hint on possible solutions to fix the above issues is highly
>> appreciated.
>>
>> Thanks.
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Issue-with-disk-space-on-UNIX-tp2676784p2676784.html
>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: java-user-h...@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
