[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964381#comment-15964381
]
James Dyer commented on SOLR-10395:
---
[~fsteggink]
bq. What does the 0 do?
I thought it was one of your query terms. So it doesn't do anything.
bq. triggering a rare condition
Its not so rare and lots of people run into problems like this. The spellcheck
field needs intact keywords like the user enters to compare user input to
possible corrections. Heavy analysis like stemming and grams break this
assumption. The spellchecker also needs to extract the actual keywords from
the query string without having access to the actual query parser that was
used. This is where the SpellingQueryConverter comes in. But the
SpellingQueryConverter has limitations. The use of this class rather than
getting the actual keywords from the real query parser is, in my opinion, a
weakness of Solr's spellchecker.
Looking a little closer, I think another way to work around this is to move the
"type" part of the query to a "fq" and just query the user's keywords in the
"q" parameter. While I can't reproduce the NPE, I can get it to suggest for
"or" and "type", which is also incorrect. Once again, the
SpellingQueryConverter assumes a query is either a boolean-type query or its a
dismax-type query. It doesn't know all the nuances of edismax.
bq. q=menweg beingen 0&fq=(type:weg OR type:adres)
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. -I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.-
> As a workaround I could live for now with turning collations off. (Update:
> I've suppressed logging of these exceptions in log4j.properties.)
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964359#comment-15964359
]
Frank Steggink commented on SOLR-10395:
---
{quote}
But an easier solution would be to use the "spellcheck.q" parameter, like this:
/suggest?q=matenweg beuningen 0(type:weg or type:adres)&spellcheck.q=matenweg
beuningen 0
{quote}
What does the 0 do?
{quote}
...but actually, for spellcheck to work well, you might need to
your data to a field that is not analyzed with ngrams but just is split on
whitespace and use that as the field you check spelling against.
{quote}
Good idea. I think I'll disable the spell check as you've originally suggested.
{quote}
The only real bug I see here is the lack of a good error message for cases like
this. The spellchecker is not meant to be used with custom query parsers or
heavily-analyzed fields, so its not surprising to me you're running into
trouble.
{quote}
Perhaps my configuration is indeed triggering a rare condition causing the
exception, but I agree that a better error message would help a lot.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. -I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.-
> As a workaround I could live for now with turning collations off. (Update:
> I've suppressed logging of these exceptions in log4j.properties.)
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964355#comment-15964355
]
Frank Steggink commented on SOLR-10395:
---
I've replaced the llparser in my search handler by edismax, in order to rule
out that my custom code was somehow responsible for this. The NPE's are still
showing up in my log files.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. -I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.-
> As a workaround I could live for now with turning collations off. (Update:
> I've suppressed logging of these exceptions in log4j.properties.)
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964336#comment-15964336
]
James Dyer commented on SOLR-10395:
---
[~fsteggink] If you're using a custom QueryParser, you probably cannot use the
default SpellingQueryConverter, but need to write one that pairs with the
QueryParser you're using.
But an easier solution would be to use the "spellcheck.q" parameter, like this:
/suggest?q=matenweg beuningen 0(type:weg or type:adres)&spellcheck.q=matenweg
beuningen 0
...but actually, for spellcheck to work well, you might need to
your data to a field that is not analyzed with ngrams but just is split on
whitespace and use that as the field you check spelling against.
The only real bug I see here is the lack of a good error message for cases like
this. The spellchecker is not meant to be used with custom query parsers or
heavily-analyzed fields, so its not surprising to me you're running into
trouble.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. -I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.-
> As a workaround I could live for now with turning collations off. (Update:
> I've suppressed logging of these exceptions in log4j.properties.)
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964298#comment-15964298
]
Frank Steggink commented on SOLR-10395:
---
{quote}
I've noticed the exception only occurs when no more than 2 alternatives are
found, but when 3 alternatives (maxCollations) are found, the NPE isn't
occurring.
{/quote}
I don't know why I said that the NPE wasn't occurring, but it still appears in
the Solr log. Also note that the collations array has always been empty.
I've also set maxCollations to 2. I see suggestions being given with 2 results
and also with 3 results, but not with 4 results. However, in all cases I'm
still seeing the NPE. I did reload the collection, after I've pushed the new
configuration to Zookeeper.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. -I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.-
> As a workaround I could live for now with turning collations off. (Update:
> I've suppressed logging of these exceptions in log4j.properties.)
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964254#comment-15964254
]
Frank Steggink commented on SOLR-10395:
---
{quote}
I've noticed the exception only occurs when no more than 2 alternatives are
found, but when 3 alternatives (maxCollations) are found, the NPE isn't
occurring.
{quote}
I don't know why I said that, but with 3 results the error is still occurring.
Not with 4 results.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.
> As a workaround I could live for now with turning collations off.
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964243#comment-15964243
]
Frank Steggink commented on SOLR-10395:
---
@James: here is an example returning only 2 results:
http://geodata.nationaalgeoregister.nl/locatieserver/suggest?q=matenweg%20beuningen%20(type:weg%20or%20type:adres)&indent=on
It still throws the error. As you can see, it also gives suggestions for
"type", "or", and "adres". This is due to synonyms, and filters in some
analyzers.
You're right that it shouldn't be necessary to use collation when using ngrams.
There might be a use case though, when someone mistypes a character. A smart
client application could do some "autocorrection" on the fly.
I'll check what happens when maxCollations is set to 2.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.
> As a workaround I could live for now with turning collations off.
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964232#comment-15964232
]
Frank Steggink commented on SOLR-10395:
---
@Matthias: here is an example request:
http://geodata.nationaalgeoregister.nl/locatieserver/suggest?q=matenweg%20beuningen&indent=on
Currently I'm suppressing the error messages on this server (3 Solr instances,
running in a Solr Cloud configuration, using external Zookeepers), but I can
still reproduce it on a test server.
Here is the full configuration of the search handler:
{code:xml}
llparser
100%
id,weergavenaam,type,score
suggest
on
suggest
255
html
on
suggest
false
5
2
3
true
true
5
3
json
*:*
10
true
exacte_match^1 suggest^0.5 huisnummer^0.5
huisletter^0.5 huisnummertoevoeging^0.5
type:gemeente^1.5 type:woonplaats^1.5 type:weg^1.5
type:postcode^0.5 type:adres^1
score desc, sortering asc, weergavenaam asc
spellcheck
{code}
We're using Solr as a geocoder. With this you can enter the name of an address,
post code, street, city/village or municipality, and you get information about
matching objects back. In this case one street, one postal code, and one
address. This street has only one address, so I'm getting 3 results, triggering
the collation, and also the exception.
Query parser:
{code:xml}
{code}
The purpose of the LatLonQueryParser is to convert custom parameters lat and
lon to parameters so Solr can execute a geospatial search. This class is based
on the ExtendedDismaxQParser. I'm just setting these new parameters (sort, pt,
boost, and distance) to the request. I'm also changing the private field config
during reflection. I've written this QP already a few years ago, for a
different client. I should probably have reported this before. All of this only
happens when the lat and lon parameters have been given. But, as far as I know,
I'm not doing anything related to spellchecking.
I'm not using any other plugins.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearc
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15961025#comment-15961025
]
James Dyer commented on SOLR-10395:
---
bq. The field is filled with lots of data created by edge ngrams, since it's
intended usage is for an autocomplete field.
The collate functionality doesn't generally work with ngrams. Before the
collate functionality existed with its current features, users would use ngram
fields to get phgrase-level spellchecking. But the collation feature is
intended to give phrase-level spell corrections without needing to index all
the additional data. So typically you'd use either ngrams _or_ you'd use
collate, but not both together. If I were you, I'd also look using one of the
Suggesters instead of spellcheck for suggestions.
Aside from this, it would be best if it didn't throw NPE! Could you give an
example of the full query parameters for a query that returns 2 results and
succeeds? Could you set "maxCollations" down from 3 to 2 and see if that
causes it to fail? If not, could you also provide an example that fails?
Possibly we can create a failing unit test, if I can know enough about what
you're doing.
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.
> As a workaround I could live for now with turning collations off.
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10395) NPE during spell check with result collation
[
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950763#comment-15950763
]
Matthias Krueger commented on SOLR-10395:
-
This is an odd code location for an NPE. I cannot see any nullable
reassignments to either {{checkResponse}}, {{checkResponse.rsp}}, or
{{checkResponse.rsp.getToLog}} in the current code base. Could you share the
search request you're sending? Do you use any other search components or
plugins in that request?
> NPE during spell check with result collation
>
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check.
> This is the message I'm getting: Exception trying to re-query to check if a
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator
> Exception trying to re-query to check if a spell check possibility would
> return any hits.
> java.lang.NullPointerException
> at
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried
> with this search handler. The field is filled with lots of data created by
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in
> the code later
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
> queryComponent.prepare(checkResponse);
> if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |=
> SolrIndexSearcher.TERMINATE_EARLY);
> }
> queryComponent.process(checkResponse);
> hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr
> 6.5.0.
> The line causing my exception is when determining the hits. I've noticed the
> exception only occurs when no more than 2 alternatives are found, but when 3
> alternatives (maxCollations) are found, the NPE isn't occurring.
> As a workaround I could live for now with turning collations off.
> Please let me know if you need more information. Thanks!
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
