Angel Todorov created SOLR-11139:
------------------------------------
Summary: FreeTextSuggester exits on first error while building
suggester index
Key: SOLR-11139
URL: https://issues.apache.org/jira/browse/SOLR-11139
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: Suggester
Affects Versions: 5.5.4
Reporter: Angel Todorov
Priority: Blocker
If an exception is thrown below, it will exit the while loop, therefore
stopping building of the suggester index. This is highly unacceptable, because
in an enterprise environment there will always be data that does not conform to
the expected rules.
while (true) {
BytesRef term = termsEnum.next();
if (term == null) {
break;
}
int ngramCount = countGrams(term);
if (ngramCount > grams) {
throw new IllegalArgumentException("tokens must not contain separator
byte; got token=" + term + " but gramCount=" + ngramCount + ", which is greater
than expected max ngram size=" + grams);
}
if (ngramCount == 1) {
totTokens += termsEnum.totalTermFreq();
}
builder.add(Util.toIntsRef(term, scratchInts),
encodeWeight(termsEnum.totalTermFreq()));
}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
