We've recently upgraded to Lucene 4.4.0 and mergeSegments now causes an
OOM error.
As background, our index contains about 14 million documents (growing
slowly) and we process about 1 million updates per day. It's about 8GB
on disk. I'm not sure if the Lucene segments merge the way they used to
in the early versions, but we've always optimized at 3am to get rid of
dead space in the index, or otherwise it grows forever.
The mergeSegments was working under 4.3.1 but the index has grown
somewhat on disk since then, probably due to a couple of added
NumericDocValues fields. The java process is assigned about 3GB (the
maximum, as it's running on a 32 bit i686 Linux box), and it still goes OOM.
Any advice as to the possible cause and how to circumvent it would be
great. Here's the stack trace:
org.apache.lucene.index.MergePolicy$MergeException:
java.lang.OutOfMemoryError: Java heap space
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:545)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.OutOfMemoryError: Java heap space
org.apache.lucene.codecs.lucene42.Lucene42DocValuesProducer.loadNumeric(Lucene42DocValuesProducer.java:212)
org.apache.lucene.codecs.lucene42.Lucene42DocValuesProducer.getNumeric(Lucene42DocValuesProducer.java:174)
org.apache.lucene.index.SegmentCoreReaders.getNormValues(SegmentCoreReaders.java:301)
org.apache.lucene.index.SegmentReader.getNormValues(SegmentReader.java:253)
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:215)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
Thanks,
Michael.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
