In a separate thread, I've detailed how an optimize is taking > 2x disk
space. We don't use solr distribution/snapshooter. We are using the
default deletion policy = 1. We can't optimize a 192G index in 400GB of
space.
This thread in lucene/java-user
http://www.gossamer-threads.com/lists/lucene/java-user/43475
suggests that an optimize should not take > 2x unless perhaps an
IndexReader is holding on to segments. This could be our problem since
when optimization runs out of space, if we stop tomcat, a number of
files go away and space is recovered.
But we are not searching the index so how could a Searcher/IndexReader
have any segments open?
I notice in the logs that as part of routine commits or as part of
optimize a Searcher is registered and autowarmed from a previous
searcher (of course there's nothing in the caches -- this is just a
build machine).
INFO: registering core:
Oct 6, 2009 2:16:20 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher searc...@2e097617 main
Does this means that there's always a lucene IndexReader holding segment
files open so they can't be deleted during an optimize so we run out of
disk space > 2x?
Phil
hathitrust.org
