GitHub user cpoerschke opened a pull request: https://github.com/apache/lucene-solr/pull/76
LUCENE-5857: genLookaheadCountLimit configurable, add genLookbackCountLimit https://issues.apache.org/jira/i#browse/LUCENE-5857 You can merge this pull request into a Git repository by running: $ git pull https://github.com/bloomberg/lucene-solr trunk-gen-look-lucene Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/76.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #76 ---- commit 47693ccdef418de0d3bb419a5071516e203a1de6 Author: Christine Poerschke <cpoersc...@bloomberg.net> Date: 2014-06-17T18:40:48Z lucene: make genLookaheadCountLimit configurable, add genLookbackCountLimit parameter The problem we saw was that an error-in-final-commit during solr-shutdown led to a corrupted segments_.... file which prevented subsequent solr-start, details below. this change: * adds genLookbackCountLimit similar to the existing genLookaheadCountLimit * makes both parameters configurable (initial values defaulted to existing behaviour) error-in-final-commit result: * segments_2fi5 file present and zero-bytes long * segments_2fi4 file absent on disk (speculation: 2fi4 was result of in-memory segment merging) * segments_2fi3 file present and usable (but not found by existing looking-logic) * segments.gen file referenced 2fi3 solr-start failure details: ERROR [coreLoadExecutor-4-thread-6] o.a.s.c.CoreContainer [CoreContainer.java:910] Unable to create core: collection_shard_replica org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_10] at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_10] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_10] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [na:1.7.0_10] at java.util.concurrent.FutureTask.run(FutureTask.java:166) [na:1.7.0_10] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_10] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_10] at java.lang.Thread.run(Thread.java:722) [na:1.7.0_10] Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838) ... 12 common frames omitted Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/dir/collection_shard_replica/data/index.YYYYMMDDHHMMSSMMM/segments_2fi5") at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:78) at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41) at org.apache.lucene.store.DataInput.readInt(DataInput.java:96) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:331) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:416) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:864) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:710) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:412) at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:749) at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64) at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267) at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110) at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1513) ... 14 common frames omitted ERROR [coreLoadExecutor-4-thread-6] o.a.s.c.CoreContainer [SolrException.java:120] null:org.apache.solr.common.SolrException: Unable to create core: collection_shard_replica at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:911) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:568) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:261) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:253) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.<init>(SolrCore.java:858) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:641) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:556) ... 10 more Caused by: org.apache.solr.common.SolrException: Error opening new searcher at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1550) at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1662) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:838) ... 12 more ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org