Index keeps growing, then shrinks on restart
Hi, I have an index that's about 700 MB, and it grows over days to until it causes problems with disk size, at about 5GB. If the JVM process ends, the index shrinks back to about 700MB, I'm calling IndexWriter.commit() all the time. What else do you call to get it to compact it's use of space? thank you, Rob
Re: Index keeps growing, then shrinks on restart
Telling us the version of lucene and the OS you're running on is always a good idea. A guess here is that you aren't closing index readers, so the JVM will be holding on to deleted files until it exits. A combination of du, ls, and lsof commands should prove it, or just losf: run it against the java process and look for deleted files. If you're on unix that is. -- Ian. On Mon, Nov 10, 2014 at 11:03 PM, Rob Nikander wrote: > Hi, > > I have an index that's about 700 MB, and it grows over days to until it > causes problems with disk size, at about 5GB. If the JVM process ends, the > index shrinks back to about 700MB, I'm calling IndexWriter.commit() all the > time. What else do you call to get it to compact it's use of space? > > thank you, > Rob - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: Index keeps growing, then shrinks on restart
On Tue, Nov 11, 2014 at 4:26 AM, Ian Lea wrote: > Telling us the version of lucene and the OS you're running on is > always a good idea. > Oops, yes. Lucene 4.10.0, Linux. A guess here is that you aren't closing index readers, so the JVM will > be holding on to deleted files until it exits. > That's probably it. I found a code path where it seems I thought the reader's `close()` would be called by GC/finalize. Rob
Re: Index keeps growing, then shrinks on restart
On Nov 10, 2014, at 3:03 PM, Rob Nikander wrote: > Hi, > > I have an index that's about 700 MB, and it grows over days to until it > causes problems with disk size, at about 5GB. If the JVM process ends, the > index shrinks back to about 700MB, I'm calling IndexWriter.commit() all the > time. What else do you call to get it to compact it's use of space? Are you maybe leaking instances of IndexReader or IndexWriter? Any deleted files that are held open by Java objects will continue to consume disk space until the Java object releases the file from use. - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
