[ https://issues.apache.org/jira/browse/LUCENE-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838132#comment-16838132 ]
Adrien Grand commented on LUCENE-8795: -------------------------------------- TestBKD (https://github.com/apache/lucene-solr/blob/master/lucene/core/src/test/org/apache/lucene/util/bkd/TestBKD.java) is the most likely test to reproduce this issue. > ArrayIndexOutOfBoundsException during System.arraycopy in BKDWriter > ------------------------------------------------------------------- > > Key: LUCENE-8795 > URL: https://issues.apache.org/jira/browse/LUCENE-8795 > Project: Lucene - Core > Issue Type: Bug > Components: core/index > Affects Versions: 6.6 > Environment: h2. Operating system/Platform > {code:java} > IBM i - V7R3M0 > {code} > h2. Java version > {code:java} > java version "1.8.0_191" > Java(TM) SE Runtime Environment (build 8.0.5.25 - > pap6480sr5fp25-20181030_01(SR5 FP25)) > IBM J9 VM (build 2.9, JRE 1.8.0 OS/400 ppc64-64-Bit Compressed References > 20181029_400846 (JIT enabled, AOT enabled) > OpenJ9 - c5c78da > OMR - 3d5ac33 > IBM - 8c1bdc2) > JCL - 20181022_01 based on Oracle jdk8u191-b26 > NOTICE: If no version information is found above, this could indicate a > corrupted Java installation! > Java detected was: /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/bin/java > -Dmultiarchive.basepath=/home/NEXTOWN/Multi-Support/Next -Xms128m -Xmx2048m > {code} > Reporter: Torben Riis > Priority: Major > > Hi, > I’m a bit stuck here and needs a clue or two in order to continue our > investigations. Hope that someone can help. :) > Periodically, around once a month, we get the below > {{ArrayIndexOutOfBoundsException}} on our system. We use multiple indexes and > the error can originate from any of them, but the error always occurs in line > 1217 in {{BKDWriter}} (during a {{System.arraycopy}}). > We found a couple of issues on the net regarding JIT optimization problem > related to J9, but they all looks like they have been resolved and cannot be > reproduced anymore. But nevertheless, we have added the following {{-Xjit}} > flags to excludes JIT optimization for every class / inner classes in the > {{bkd}} package. Moreover, we have also made a complete copy of the whole > installation in production, and added the opposite arguments (enforce JIT > optimizations for the specific classes). First we will try with > {{optLevel=hot}}, but if this doesn’t show anything we will afterwards try > with {{veryHot}} and {{scorching}}. Unfortunately we do not have the result > of this yet, but I’ll of course post it when it is known. > Unfortunately it’s not possible to run OpenJDK on the IBM i platform, so such > a test will not be possible, but it is worth mentioning that our product is a > standard product, which typically run on the Windows or Linux platform using > AdoptOpenJDK. Currently we have a couple of hundred installations running out > there on these platforms, and without any problems. But on the IBM I platform > with J9 we sometimes see this exception. > Any good ideas for further investigations? Or have seen such issue before? > We are using Lucene 6.6.0 and runs on IBM J9 on the IBM I platform. > h2. Stacktrace > {code:java} > Exception in thread "Lucene Merge Thread #0" 2019-05-01T06:10:07.970 CEST > [Lucene Merge Thread #0] org.apache.lucene.index.MergePolicy$MergeException: > java.lang.ArrayIndexOutOfBoundsException > at > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:683) > Caused by: 2019-05-01T06:10:07.971 CEST [Lucene Merge Thread #0] > java.lang.ArrayIndexOutOfBoundsException > at > org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1217) > at > org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1197) > at > org.apache.lucene.util.bkd.BKDWriter.packIndex(BKDWriter.java:1078) > at > org.apache.lucene.util.bkd.BKDWriter.writeIndex(BKDWriter.java:1245) > at > org.apache.lucene.util.bkd.BKDWriter.access$600(BKDWriter.java:82) > at > org.apache.lucene.util.bkd.BKDWriter$OneDimensionBKDWriter.finish(BKDWriter.java:648) > at org.apache.lucene.util.bkd.BKDWriter.merge(BKDWriter.java:560) > at > org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:212) > at > org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173) > at > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356) > at > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:661) > Exception in thread "Lucene Merge Thread #0" 2019-05-01T06:10:08.075 CEST > [Lucene Merge Thread #0] org.apache.lucene.index.MergePolicy$MergeException: > java.lang.ArrayIndexOutOfBoundsException > at > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:703) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:683) > Caused by: 2019-05-01T06:10:08.076 CEST [Lucene Merge Thread #0] > java.lang.ArrayIndexOutOfBoundsException > at > org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1217) > at > org.apache.lucene.util.bkd.BKDWriter.recursePackIndex(BKDWriter.java:1197) > at > org.apache.lucene.util.bkd.BKDWriter.packIndex(BKDWriter.java:1078) > at > org.apache.lucene.util.bkd.BKDWriter.writeIndex(BKDWriter.java:1245) > at > org.apache.lucene.util.bkd.BKDWriter.access$600(BKDWriter.java:82) > at > org.apache.lucene.util.bkd.BKDWriter$OneDimensionBKDWriter.finish(BKDWriter.java:648) > at org.apache.lucene.util.bkd.BKDWriter.merge(BKDWriter.java:560) > at > org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:212) > at > org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:173) > at > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:122) > at > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356) > at > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931) > at > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:624) > at > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler > {code} > h2. On production we have disabled jit optimization for the following classes > {code:java} > -Xjit:verbose,exclude={org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDReader.*|org/apache/lucene/util/bkd/BKDReader$IndexTree.*|org/apache/lucene/util/bkd/BKDReader$IntersectState.*|org/apache/lucene/util/bkd/BKDReader$LegacyIndexTree.*|org/apache/lucene/util/bkd/BKDReader$PackedIndexTree.*|org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$OneDimensionBKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$MergeReader.*|org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.*|org/apache/lucene/util/bkd/BKDWriter$PathSlice.*|org/apache/lucene/util/bkd/DocIdsWriter.*|org/apache/lucene/util/bkd/HeapPointReader.*|org/apache/lucene/util/bkd/HeapPointWriter.*|org/apache/lucene/util/bkd/MutablePointsReaderUtils.*|org/apache/lucene/util/bkd/OfflinePointReader.*|org/apache/lucene/util/bkd/OfflinePointWriter.*|org/apache/lucene/util/bkd/PointReader.*|org/apache/lucene/util/bkd/PointWriter.*},vlog=/home/nextown/multi-support/next/logs/jit > {code} > h2. On the copy of production we have enforced jit optimization with the > following arguments > {code:java} > -Xjit:{org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDReader.*|org/apache/lucene/util/bkd/BKDReader$IndexTree.*|org/apache/lucene/util/bkd/BKDReader$IntersectState.*|org/apache/lucene/util/bkd/BKDReader$LegacyIndexTree.*|org/apache/lucene/util/bkd/BKDReader$PackedIndexTree.*|org/apache/lucene/util/bkd/BKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$OneDimensionBKDWriter.*|org/apache/lucene/util/bkd/BKDWriter$MergeReader.*|org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.*|org/apache/lucene/util/bkd/BKDWriter$PathSlice.*|org/apache/lucene/util/bkd/DocIdsWriter.*|org/apache/lucene/util/bkd/HeapPointReader.*|org/apache/lucene/util/bkd/HeapPointWriter.*|org/apache/lucene/util/bkd/MutablePointsReaderUtils.*|org/apache/lucene/util/bkd/OfflinePointReader.*|org/apache/lucene/util/bkd/OfflinePointWriter.*|org/apache/lucene/util/bkd/PointReader.*|org/apache/lucene/util/bkd/PointWriter.*}(count=0,optlevel=hot) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org