IBM's J9 JVM unfortunately still has a number of nasty bugs affecting Lucene; most likely you are hitting one of these. We used to test J9 in our continuous Jenkins jobs, but there were just too many J9-specific failures and we couldn't get IBM's attention to resolve them, so we stopped. For now you should switch to Oracle JDK, or OpenJDK.
But there's some good news! Recently, a member from the IBM JDK team replied to this Elasticsearch thread: https://discuss.elastic.co/t/need-help-with-ibm-jdk-issues-with-es-1-4-5/1748/3 And then Robert Muir ran Lucene's tests with the latest J9 and opened several issues; see the 2nd bullet under Apache Lucene at https://www.elastic.co/blog/this-week-in-elasticsearch-and-apache-lucene-2015-06-09 and at least one of the issues seems to be making progress (https://issues.apache.org/jira/browse/LUCENE-6522). So there is hope for the future, but for today it's too dangerous to use J9 with Lucene/Solr/Elasticsearch. Mike McCandless http://blog.mikemccandless.com On Tue, Jun 9, 2015 at 12:23 PM, Guy Moshkowich <g...@il.ibm.com> wrote: > We are using Solr 4.7.2 and we found that when we run > CheckIndex.checkIndex on one of the Solr shards we are getting the error > below. > Both replicas of the shard had the same error. > The shard index looked healthy: > 1) It appeared active in the Solr admin page. > 2) We could run searches against it. > 3) No relevant errors where found in Solr logs. > 4) After we optimized the index in LUKE, CheckIndex did not report any > error. > > My questions: > 1) Is this is a real issue or a known bug in CheckIndex code that cause > false negative ? > 2) Is there a known fix for this issue? > > Here is the error we got: > validateIndex Segments file=segments_bhe numSegments=15 version=4.7 > format= userData={commitTimeMSec=1432689607801} > 1 of 15: name=_6cth docCount=248744 > codec=Lucene46 > compound=false > numFiles=11 > size (MB)=86.542 > diagnostics = {timestamp=1428883354605, os=Linux, > os.version=2.6.32-431.23.3.el6.x86_64, mergeFactor=10, source=merge, > lucene.version=4.7.2 1586229 - rmuir - 2014-04-10 09:00:35, os.arch=amd64, > mergeMaxNumSegments=-1, java.version=1.7.0, java.vendor=IBM Corporation} > has deletions [delGen=3174] > test: open reader.........FAILED > WARNING: fixIndex() would remove reference to this segment; full > exception: > java.lang.RuntimeException: liveDocs count mismatch: info=156867, vs > bits=156872 > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:581) > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:372) > > Appreciate yout help, > Guy.
