[ https://issues.apache.org/jira/browse/LUCENE-7440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15478450#comment-15478450 ]
Uwe Schindler commented on LUCENE-7440: --------------------------------------- +1 compareUnsigned is the way to go! It's an intrinsic, too. > Document skipping on large indexes is broken > -------------------------------------------- > > Key: LUCENE-7440 > URL: https://issues.apache.org/jira/browse/LUCENE-7440 > Project: Lucene - Core > Issue Type: Bug > Components: core/search > Affects Versions: 2.2 > Reporter: Yonik Seeley > Priority: Critical > Attachments: LUCENE-7440.patch > > > Large skips on large indexes fail. > Anything that uses skips (such as a boolean query, filtered queries, faceted > queries, join queries, etc) can trigger this bug on a sufficiently large > index. > The bug is a numeric overflow in MultiLevelSkipList that has been present > since inception (Lucene 2.2). It may not manifest until one has a single > segment with more than ~1.8B documents, and a large skip is performed on that > segment. > Typical stack trace on Lucene7-dev: > {code} > java.lang.ArrayIndexOutOfBoundsException: 110 > at > org.apache.lucene.codecs.MultiLevelSkipListReader$SkipBuffer.readByte(MultiLevelSkipListReader.java:297) > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125) > at > org.apache.lucene.codecs.lucene50.Lucene50SkipReader.readSkipData(Lucene50SkipReader.java:180) > at > org.apache.lucene.codecs.MultiLevelSkipListReader.loadNextSkip(MultiLevelSkipListReader.java:163) > at > org.apache.lucene.codecs.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:133) > at > org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockDocsEnum.advance(Lucene50PostingsReader.java:421) > at YCS_skip7$1.testSkip(YCS_skip7.java:307) > {code} > Typical stack trace on Lucene4.10.3: > {code} > 6-08-31 18:57:17,460 ERROR org.apache.solr.servlet.SolrDispatchFilter: > null:java.lang.ArrayIndexOutOfBoundsException: 75 > at > org.apache.lucene.codecs.MultiLevelSkipListReader$SkipBuffer.readByte(MultiLevelSkipListReader.java:301) > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:122) > at > org.apache.lucene.codecs.lucene41.Lucene41SkipReader.readSkipData(Lucene41SkipReader.java:194) > at > org.apache.lucene.codecs.MultiLevelSkipListReader.loadNextSkip(MultiLevelSkipListReader.java:168) > at > org.apache.lucene.codecs.MultiLevelSkipListReader.skipTo(MultiLevelSkipListReader.java:138) > at > org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.advance(Lucene41PostingsReader.java:506) > at org.apache.lucene.search.TermScorer.advance(TermScorer.java:85) > [...] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621) > [...] > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2004) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org