This is reproducible only within our product, I haven't yet been able to
isolate this and reproduce it standalone. It's Java 11.
Yes, I've run CheckIndex with the "-slow" option and with assertions
enabled.
El 24/11/20 a las 11:32, Adrien Grand escribió:
This is related to phrase matching indeed. Positions are stored in
blocks of 128 values, where every block is encoded with a different
number of bits per value. And the error you are seeing suggests that
one block reports 69 bits per value.
The fact that CheckIndex didn't complain is surprising. Did you only
verify checksums (the -fast option) or did you run the full CheckIndex?
Is your problem reproducible? If yes, does it still reproduce if you
move to a recent JVM?
On Tue, Nov 24, 2020 at 3:22 PM Nicolás Lichtmaier
<nicol...@wolfram.com <mailto:nicol...@wolfram.com>> wrote:
Lucene 8.7's CheckIndex says there are no errors in the index.
On closer inspection this seems related to phrase matching...
El 24/11/20 a las 05:18, Adrien Grand escribió:
> Can you run CheckIndex on your index to make sure it is not corrupt?
>
> On Tue, Nov 24, 2020 at 1:01 AM Nicolás Lichtmaier
> <nicol...@wolfram.com.invalid> wrote:
>
>> I'm seeing errors like this one (using backwards codecs):
>>
>> java.lang.ArrayIndexOutOfBoundsException: Index 69 out of
bounds for
>> length 33
>> at
>>
org.apache.lucene.codecs.lucene50.ForUtil.readBlock(ForUtil.java:196)
>> at
>>
>>
org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$EverythingEnum.refillPositions(Lucene50PostingsReader.java:721)
>> at
>>
>>
org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$EverythingEnum.nextPosition(Lucene50PostingsReader.java:924)
>> at
>>
>>
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
>> at
>>
>>
org.apache.lucene.search.SloppyPhraseMatcher.advancePP(SloppyPhraseMatcher.java:262)
>> at
>>
>>
org.apache.lucene.search.SloppyPhraseMatcher.nextMatch(SloppyPhraseMatcher.java:173)
>> at
>>
org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:58)
>> at
>>
>>
org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
>> at
>>
>>
org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
>> at
>>
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
>> at
>>
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
>> at
>>
>>
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
>> at
>>
>>
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
>> at
>>
>>
org.apache.lucene.search.DisjunctionMaxScorer.score(DisjunctionMaxScorer.java:67)
>> at
>>
>>
org.apache.lucene.search.DisjunctionScorer.score(DisjunctionScorer.java:194)
>> at
>>
>>
org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
>> at
>>
>>
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
>> at
>>
>>
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
>> at
>>
>>
org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
>> at
>>
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
>> at
>>
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
>> at
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
>> at
>>
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:661)
>> at
>>
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)
>> at
>>
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:574)
>> at
>>
org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:421)
>> at
>>
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:432)
>>
>> They seem to be connected with double values stored as
"docvalues" and
>> user in formulas to affect the scores.
>>
>> Is there any known incompatibility? Is this something that
should work?
>> Must I rebuild the indices with 8.7? (that would be bad for our
usecase
>> here)
>>
>> Thanks!
>>
>>
>>
---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
<mailto:java-user-unsubscr...@lucene.apache.org>
>> For additional commands, e-mail:
java-user-h...@lucene.apache.org
<mailto:java-user-h...@lucene.apache.org>
>>
>>
--
Adrien
