On 10/28/2015 2:06 PM, Rallavagu wrote: > Solr 4.6.1, cloud > > Seeing following commit errors. > > [commitScheduler-19-thread-1] ERROR > org.apache.solr.update.CommitTracker – auto commit > error...:java.lang.IllegalStateException: this writer hit an > OutOfMemoryError; cannot commit at > org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2807) > at > org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2984) > at > org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:559) > at org.apache.solr.update.CommitTracker.run(CommitTracker.java:216) at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:440) at > java.util.concurrent.FutureTask.run(FutureTask.java:138) at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:896) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > at java.lang.Thread.run(Thread.java:682) > > Looking at the code, > > public final void prepareCommit() throws IOException { > ensureOpen(); > prepareCommitInternal(); > } > > private void prepareCommitInternal() throws IOException { > synchronized(commitLock) { > ensureOpen(false); > if (infoStream.isEnabled("IW")) { > infoStream.message("IW", "prepareCommit: flush"); > infoStream.message("IW", " index before flush " + segString()); > } > > if (hitOOM) { > throw new IllegalStateException("this writer hit an > OutOfMemoryError; cannot commit"); > } > > It simply checking a flag if it hit OOM? What is making to check and > set the flag? What could be the conditions? Thanks.
This exception handling was revamped in Lucene 4.10.1 (and therefore in Solr 4.10.1) by this issue: https://issues.apache.org/jira/browse/LUCENE-5958 The "hitOOM" variable was removed by the following specific commit -- this is the commit on the 4.10 branch, but it was also committed to branch_4x and trunk as well. Later commits on this same issue were made to branch_5x -- the cutover to begin the 5.0 release process was made while this issue was still being fixed. https://svn.apache.org/viewvc/lucene/dev/branches/lucene_solr_4_10/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java?r1=1626189&r2=1626188&pathrev=1626189 In the code before this fix, the hitOOM flag is set by other methods in IndexWriter. It is volatile to prevent problems with multiple threads updating and accessing it. Your message doesn't indicate what problems you're having besides an error message in your log. LUCENE-5958 indicates that the problems could be as bad as a corrupt index. The reason that IndexWriter swallows OOM exceptions is that this is the only way Lucene can even *attempt* to avoid index corruption in every error situation. Lucene has had a very good track record at avoiding index corruption, but every now and then a bug is found and a user manages to get a corrupted index. Thanks, Shawn