[jira] [Updated] (LUCENE-8185) HyphenationCompoundWordTokenFilter returns terms shorter than minSubwordSize

2018-03-06 Thread Matthias Krueger (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Krueger updated LUCENE-8185:
-
Affects Version/s: 6.6.1
   7.2.1

> HyphenationCompoundWordTokenFilter returns terms shorter than minSubwordSize
> 
>
> Key: LUCENE-8185
> URL: https://issues.apache.org/jira/browse/LUCENE-8185
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.6.1, 7.2.1
>Reporter: Matthias Krueger
>Priority: Minor
> Attachments: LUCENE-8185.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> To account for languages which use binding characters ("fogemorphemes") for 
> composing words the {{HyphenationCompoundWordTokenFilter}} re-checks the 
> dictionary for a candidate with the last character removed when the original 
> candidate was not found. It currently does not re-check against 
> {{minSubWordSize}} in this case. Terms that are one character shorter than 
> {{minSubWordSize}} can be returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-8183) HyphenationCompoundWordTokenFilter creates overlapping tokens with onlyLongestMatch enabled

2018-02-26 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376826#comment-16376826
 ] 

Matthias Krueger commented on LUCENE-8183:
--

You might want to have a look at "mocking" the HyphenationTree (see my patch 
for LUCENE-8185) which simplifies writing a decompounding test.

> HyphenationCompoundWordTokenFilter creates overlapping tokens with 
> onlyLongestMatch enabled
> ---
>
> Key: LUCENE-8183
> URL: https://issues.apache.org/jira/browse/LUCENE-8183
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Affects Versions: 6.6
> Environment: Configuration of the analyzer:
> 
> 
>          hyphenator="lang/hyph_de_DR.xml" encoding="iso-8859-1"
>          dictionary="lang/wordlist_de.txt" 
>         onlyLongestMatch="true"/>
>  
>Reporter: Rupert Westenthaler
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: LUCENE-8183_20180223_rwesten.diff, lucene-8183.zip
>
>
> The HyphenationCompoundWordTokenFilter creates overlapping tokens even if 
> onlyLongestMatch is enabled. 
> Example:
> Dictionary: {{gesellschaft}}, {{schaft}}
>  Hyphenator: {{de_DR.xml}} //from Apche Offo
>  onlyLongestMatch: true
>  
> |text|gesellschaft|gesellschaft|schaft|
> |raw_bytes|[67 65 73 65 6c 6c 73 63 68 61 66 74]|[67 65 73 65 6c 6c 73 63 68 
> 61 66 74]|[73 63 68 61 66 74]|
> |start|0|0|0|
> |end|12|12|12|
> |positionLength|1|1|1|
> |type|word|word|word|
> |position|1|1|1|
> IMHO this includes 2 unexpected Tokens
>  # the 2nd 'gesellschaft' as it duplicates the original token
>  # the 'schaft' as it is a sub-token 'gesellschaft' that is present in the 
> dictionary
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-8185) HyphenationCompoundWordTokenFilter returns terms shorter than minSubwordSize

2018-02-25 Thread Matthias Krueger (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Krueger updated LUCENE-8185:
-
Attachment: LUCENE-8185.patch

> HyphenationCompoundWordTokenFilter returns terms shorter than minSubwordSize
> 
>
> Key: LUCENE-8185
> URL: https://issues.apache.org/jira/browse/LUCENE-8185
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Matthias Krueger
>Priority: Minor
> Attachments: LUCENE-8185.patch
>
>
> To account for languages which use binding characters ("fogemorphemes") for 
> composing words the {{HyphenationCompoundWordTokenFilter}} re-checks the 
> dictionary for a candidate with the last character removed when the original 
> candidate was not found. It currently does not re-check against 
> {{minSubWordSize}} in this case. Terms that are one character shorter than 
> {{minSubWordSize}} can be returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-8185) HyphenationCompoundWordTokenFilter returns terms shorter than minSubwordSize

2018-02-25 Thread Matthias Krueger (JIRA)
Matthias Krueger created LUCENE-8185:


 Summary: HyphenationCompoundWordTokenFilter returns terms shorter 
than minSubwordSize
 Key: LUCENE-8185
 URL: https://issues.apache.org/jira/browse/LUCENE-8185
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Matthias Krueger


To account for languages which use binding characters ("fogemorphemes") for 
composing words the {{HyphenationCompoundWordTokenFilter}} re-checks the 
dictionary for a candidate with the last character removed when the original 
candidate was not found. It currently does not re-check against 
{{minSubWordSize}} in this case. Terms that are one character shorter than 
{{minSubWordSize}} can be returned.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-8183) HyphenationCompoundWordTokenFilter creates overlapping tokens with onlyLongestMatch enabled

2018-02-23 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-8183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374886#comment-16374886
 ] 

Matthias Krueger commented on LUCENE-8183:
--

[~rwesten] Quick question regarding your patch: What's the reasoning behind not 
decomposing terms that are part of the dictionary at all?

The {{onlyLongestMatch}} flag currently affects whether all matches or only the 
longest match should be returned *per* *start* character (in 
DictionaryCompoundWordTokenFilter) or *per* hyphenation *start* point (in 
HyphenationCompoundWordTokenFilter).

Example:
 Dictionary {{"Schaft", "Wirt", "Wirtschaft", "Wissen", "Wissenschaft"}} for 
input "Wirtschaftswissenschaft" will return the original input plus tokens 
"Wirtschaft", "schaft", "wissenschaft", "schaft" but not "Wirt" or "Wissen". 
"schaft" is still returned (even twice) because it's the longest token starting 
at the respective position.

I like the idea of restricting this further to only the longest terms that 
*touch* a certain hyphenation point. This would exclude "schaft" in the example 
above (as "Wirtschaft" and "wissenschaft" are two longer terms encompassing the 
respective hyphenation point). On the other hand, there might be examples where 
you still want to include the "overlapping" tokens. For "Fußballpumpe" and 
dictionary {{"Ball", "Ballpumpe", "Pumpe", "Fuß", "Fußball"}} you would get the 
tokens "Fußball" and "pumpe" but not "Ballpumpe" as "Ball" has already been 
considered part of Fußball. Also, not sure if your change also improves the 
situation for languages other than German.

Regarding point 1: The current algorithm always returns the term itself again 
if it's part of the dictionary. I guess, this could be changed if we don't 
check against {{this.maxSubwordSize}} but against 
{{Math.min(this.maxSubwordSize), termAtt.length()-1)}}

Perhaps these kind of adjustments should rather be done in a TokenFilter 
similar to RemoveDuplicatesTokenFilter instead of complicating the 
decompounding algorithm?

> HyphenationCompoundWordTokenFilter creates overlapping tokens with 
> onlyLongestMatch enabled
> ---
>
> Key: LUCENE-8183
> URL: https://issues.apache.org/jira/browse/LUCENE-8183
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/analysis
>Affects Versions: 6.6
> Environment: Configuration of the analyzer:
> 
> 
>          hyphenator="lang/hyph_de_DR.xml" encoding="iso-8859-1"
>          dictionary="lang/wordlist_de.txt" 
>         onlyLongestMatch="true"/>
>  
>Reporter: Rupert Westenthaler
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: LUCENE-8183_20180223_rwesten.diff
>
>
> The HyphenationCompoundWordTokenFilter creates overlapping tokens even if 
> onlyLongestMatch is enabled. 
> Example:
> Dictionary: {{gesellschaft}}, {{schaft}}
>  Hyphenator: {{de_DR.xml}} //from Apche Offo
>  onlyLongestMatch: true
>  
> |text|gesellschaft|gesellschaft|schaft|
> |raw_bytes|[67 65 73 65 6c 6c 73 63 68 61 66 74]|[67 65 73 65 6c 6c 73 63 68 
> 61 66 74]|[73 63 68 61 66 74]|
> |start|0|0|0|
> |end|12|12|12|
> |positionLength|1|1|1|
> |type|word|word|word|
> |position|1|1|1|
> IMHO this includes 2 unexpected Tokens
>  # the 2nd 'gesellschaft' as it duplicates the original token
>  # the 'schaft' as it is a sub-token 'gesellschaft' that is present in the 
> dictionary
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-11974) TestCollapseQParserPlugin.testStringCollapse() failure

2018-02-15 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365488#comment-16365488
 ] 

Matthias Krueger commented on SOLR-11974:
-

[~joel.bernstein] FYI Not sure if this is related but there are several bugs 
open due to (intentional or not) chaining of CollapsingQParserPlugins. See 
SOLR-8807, SOLR-9104, SOLR-10336, SOLR-10885, SOLR-7435.

> TestCollapseQParserPlugin.testStringCollapse() failure
> --
>
> Key: SOLR-11974
> URL: https://issues.apache.org/jira/browse/SOLR-11974
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Steve Rowe
>Priority: Major
>
> Reproducing branch_7x seed from my Jenkins - reproduces for me on master too:
> {noformat}
> Checking out Revision 6ae6ce2f42b61b117c31149ac87b1cb3d6ffe440 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestCollapseQParserPlugin -Dtests.method=testStringCollapse 
> -Dtests.seed=32531513E27A1FB -Dtests.slow=true -Dtests.locale=fr-LU 
> -Dtests.timezone=Europe/Warsaw -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.15s J5  | TestCollapseQParserPlugin.testStringCollapse 
> <<<
>[junit4]> Throwable #1: java.lang.AssertionError
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([32531513E27A1FB:F6AD0130F74002EF]:0)
>[junit4]>  at 
> org.apache.lucene.index.AssertingLeafReader$AssertingSortedDocValues.advanceExact(AssertingLeafReader.java:617)
>[junit4]>  at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.collect(CollapsingQParserPlugin.java:557)
>[junit4]>  at 
> org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:62)
>[junit4]>  at 
> org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
>[junit4]>  at 
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:660)
>[junit4]>  at 
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
>[junit4]>  at 
> org.apache.solr.search.Grouping.searchWithTimeLimiter(Grouping.java:447)
>[junit4]>  at 
> org.apache.solr.search.Grouping.execute(Grouping.java:395)
>[junit4]>  at 
> org.apache.solr.handler.component.QueryComponent.doProcessGroupedSearch(QueryComponent.java:1409)
>[junit4]>  at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:367)
>[junit4]>  at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
>[junit4]>  at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
>[junit4]>  at 
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
>[junit4]>  at 
> org.apache.solr.util.TestHarness.query(TestHarness.java:337)
>[junit4]>  at 
> org.apache.solr.util.TestHarness.query(TestHarness.java:319)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:876)
>[junit4]>  at 
> org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:862)
>[junit4]>  at 
> org.apache.solr.search.TestCollapseQParserPlugin.testCollapseQueries(TestCollapseQParserPlugin.java:768)
>[junit4]>  at 
> org.apache.solr.search.TestCollapseQParserPlugin.testStringCollapse(TestCollapseQParserPlugin.java:213)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> [...]
>[junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
> {text_ws=FSTOrd50, term_s=PostingsFormat(name=Memory), 
> id=PostingsFormat(name=MockRandom), group_s=PostingsFormat(name=MockRandom), 
> group_s_dv=FSTOrd50}, docValues:{id_i=DocValuesFormat(name=Direct), 
> group_i=DocValuesFormat(name=Direct), 
> group_ti_dv=DocValuesFormat(name=Lucene70), 
> _version_=DocValuesFormat(name=Asserting), 
> test_l=DocValuesFormat(name=Direct), 
> group_tf_dv=DocValuesFormat(name=Memory), 
> test_i=DocValuesFormat(name=Lucene70), 
> group_f=DocValuesFormat(name=Asserting), test_f=DocValuesFormat(name=Memory), 
> group_s_dv=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1995, 
> maxMBSortInHeap=7.1608710565771645, sim=RandomSimilarity(queryNorm=false): 
> {}, locale=fr-LU, timezone=Europe/Warsaw
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_151 (64-bit)/cpus=16,threads=1,free=168356360,total=529530880
> {noformat}
> I see this elsewhere in the log, multiple times:
> {noformat}
>[junit4]   2> 484673 ERROR 
> (TEST-TestCollapseQParserPlugin.testNumericCollapse-seed#[32531513E27A1FB]) [ 
>] o.a.s.h.RequestHandlerBase java.lang.Runtim

[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model

2017-07-31 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107364#comment-16107364
 ] 

Matthias Krueger commented on SOLR-11049:
-

See https://lucene.apache.org/solr/guide/6_6/blob-store-api.html or 
https://cwiki.apache.org/confluence/display/solr/Blob+Store+API.

> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about 
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store' 
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
>   "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the 
> model is not visible and when I try to run a query that uses the model I get 
> the following error
> {code:html}
> "error":{
> "metadata":[
>   "error-class","org.apache.solr.common.SolrException",
>   "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size 
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
>   "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without 
> error.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-9104) NPE in CollapsingQParser when two fq={!collapse} and zero results

2017-06-02 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-9104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16034358#comment-16034358
 ] 

Matthias Krueger commented on SOLR-9104:


[~markus.jel...@openindex.io]: See comment from [~hossman] here or mine on 
SOLR-8807. It seems that two Collapse-Filters just don't work if there are 
documents returned from less than all segments. A quick fix seems to be to call 
{{super.doSetNextReader(context)}} as first thing in all of 
CollapsingQParserPlugin's DelegatingCollectors' {{doSetNextReaders}} but I 
don't know if this has any undesired side effects.

> NPE in CollapsingQParser when two fq={!collapse} and zero results
> -
>
> Key: SOLR-9104
> URL: https://issues.apache.org/jira/browse/SOLR-9104
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Markus Jelsma
> Fix For: 6.2, master (7.0)
>
>
> This is a very weird problem that is reproducible on a small production 
> server, but not on the local machine although they run the same 6.0 version., 
> and have an almost identical index. The only minor difference is that 
> production is a SolrCloud with 1 shard and two replica's, just for a bit of 
> redundancy.
> The following query yields zero results and throws the NPE:
> {code}
> select?q=query:seis&fq={!collapse field=query_digest}&fq={!collapse 
> field=result_digest}
> {code}
> The next query does yield results and does not throw anything, it just works 
> as it should work:
> {code}
> select?q=query:seiz&fq={!collapse field=query_digest}&fq={!collapse 
> field=result_digest}
> {code}
> The /select handler used does not add any fancy param other than rows.
> Here's the NPE:
> {code}
> 2016-05-11 14:10:27.666 ERROR (qtp1209271652-3338) [c:suggestions s:shard1 
> r:core_node1 x:suggestions_shard1_replica1] o.a.s.s.HttpSolrCall 
> null:java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:814)
> at 
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:851)
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:272)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1794)
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1611)
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:634)
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:529)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:287)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
> {code}
> Edit: for the sake of clarity. It really needs double fq={!collapse bla bla 
> for the NPE to appear. If i remove either of the filters from the query, i 
> get a nice zero resultset back. Both fields are defined as int.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-8807) NPE during spell checking when result collapsing is activated and index got more than one segment.

2017-04-07 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15960948#comment-15960948
 ] 

Matthias Krueger commented on SOLR-8807:


Thanks, [~jdyer], looks good to me. This won't help the linked 
CollapsingQParser issues (which all seem to have the same root cause) but will 
solve the SpellCheckCollator's problem.

> NPE during spell checking when result collapsing is activated and index got 
> more than one segment.
> --
>
> Key: SOLR-8807
> URL: https://issues.apache.org/jira/browse/SOLR-8807
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 5.4
> Environment: Solr 5.4 with an index cosisting of two segments
>Reporter: Christian Danninger
>Priority: Critical
>  Labels: collapse, spellcheck
> Attachments: SOLR-8806-failing-unit-test.patch, SOLR-8807.patch, 
> SOLR-8807.patch
>
>
> When using spellchecker with collapse/expand results, I got an NPE. Only 
> happend when the index consists of more than one segment. 
> {code}
> 11:30:33,505 WARN  [org.apache.solr.spelling.SpellCheckCollator] 
> (http-/0.0.0.0:8080-2) Exception trying to re-query to check if a spell check 
> possibility would return any hits.: java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:631)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:681)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:213)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:147)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:238)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:203)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:281)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> {code}
> {code}
>   
>   
> spellchecker
>   
> 
> {code}
> The query parameters are:
> {code}
>   "spellcheck.maxCollations": "5",
>   "q.op": "AND",
>   "fq": "{!collapse field=type}",
>   "spellcheck.maxCollationTries": "10",
>   "spellcheck.collateMaxCollectDocs": "10",
>   "spellcheck.alternativeTermCount": "10",
>   "spellcheck.extendedResults": "true",
>   "spellcheck.dictionary": [
> "dest_wordbreak",
> "dest_fuzzy"
>   ],
>   "q": "kosamui thailand",
>   "defType": "edismax",
>   "expand": "true",
>   "spellcheck.maxResultsForSuggest": "3",
>   "qf": "country_name region_name",
>   "spellcheck": "true",
>   "spellcheck.accuracy": "0.5",
>   "spellcheck.count": "20",
>   "spellcheck.collate": "true",
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-10395) NPE during spell check with result collation

2017-03-31 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950763#comment-15950763
 ] 

Matthias Krueger commented on SOLR-10395:
-

This is an odd code location for an NPE. I cannot see any nullable 
reassignments to either {{checkResponse}}, {{checkResponse.rsp}}, or 
{{checkResponse.rsp.getToLog}} in the current code base. Could you share the 
search request you're sending? Do you use any other search components or 
plugins in that request?


> NPE during spell check with result collation
> 
>
> Key: SOLR-10395
> URL: https://issues.apache.org/jira/browse/SOLR-10395
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: spellchecker
>Affects Versions: 6.2.1
>Reporter: Frank Steggink
>
> I'm getting a NullPointerException when using collation during spell check. 
> This is the message I'm getting: Exception trying to re-query to check if a 
> spell check possibility would return any hits.
> Stack trace:
> {code}
> 2017-03-13 13:05:54.618 WARN (qtp472654579-84) [c:locatieserver s:dkk 
> r:core_node4 x:locatieserver_dkk_replica2] o.a.s.s.SpellCheckCollator 
> Exception trying to re-query to check if a spell check possibility would 
> return any hits.
> java.lang.NullPointerException
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:150)
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:297)
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:209)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
> ...
> {code}
> My spell check config for this search handler:
> {code:xml}
> on
> suggest
> false
> 5
> 2
> 3
> true
> true
> 5
> 3
> {code}
> The field "suggest" is a field in the index, which is also (usually) queried 
> with this search handler. The field is filled with lots of data created by 
> edge ngrams, since it's intended usage is for an autocomplete field.
> This issue is very similar to SOLR-8807, but the NPE is occurring one line in 
> the code later 
> (solr\core\src\java\org\apache\solr\spelling\SpellCheckCollator.java). As a 
> result the patch attached to that issue probably won't work in my case.
> {code:java}
> try {
>   queryComponent.prepare(checkResponse);
>   if (docCollectionLimit > 0) {
> int f = checkResponse.getFieldFlags();
> checkResponse.setFieldFlags(f |= 
> SolrIndexSearcher.TERMINATE_EARLY);
>   }
>   queryComponent.process(checkResponse);
>   hits = (Integer) checkResponse.rsp.getToLog().get("hits");
> } catch (EarlyTerminatingCollectorException etce) {
> {code}
> Although I'm using Solr 6.2.1, the source file is still identical to Solr 
> 6.5.0.
> The line causing my exception is when determining the hits. I've noticed the 
> exception only occurs when no more than 2 alternatives are found, but when 3 
> alternatives (maxCollations) are found, the NPE isn't occurring.
> As a workaround I could live for now with turning collations off.
> Please let me know if you need more information. Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Comment Edited] (SOLR-8807) NPE during spell checking when result collapsing is activated and index got more than one segment.

2017-03-21 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935520#comment-15935520
 ] 

Matthias Krueger edited comment on SOLR-8807 at 3/21/17 11:35 PM:
--

This seems to be very much related to SOLR-9104, SOLR-10336, and SOLR-7435 or 
even the same cause.


was (Author: mkrio):
This seems to be very much related to SOLR-9104 and SOLR-10336 or even the same 
cause.

> NPE during spell checking when result collapsing is activated and index got 
> more than one segment.
> --
>
> Key: SOLR-8807
> URL: https://issues.apache.org/jira/browse/SOLR-8807
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 5.4
> Environment: Solr 5.4 with an index cosisting of two segments
>Reporter: Christian Danninger
>Priority: Critical
>  Labels: collapse, spellcheck
> Attachments: SOLR-8807.patch
>
>
> When using spellchecker with collapse/expand results, I got an NPE. Only 
> happend when the index consists of more than one segment. 
> {code}
> 11:30:33,505 WARN  [org.apache.solr.spelling.SpellCheckCollator] 
> (http-/0.0.0.0:8080-2) Exception trying to re-query to check if a spell check 
> possibility would return any hits.: java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:631)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:681)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:213)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:147)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:238)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:203)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:281)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> {code}
> {code}
>   
>   
> spellchecker
>   
> 
> {code}
> The query parameters are:
> {code}
>   "spellcheck.maxCollations": "5",
>   "q.op": "AND",
>   "fq": "{!collapse field=type}",
>   "spellcheck.maxCollationTries": "10",
>   "spellcheck.collateMaxCollectDocs": "10",
>   "spellcheck.alternativeTermCount": "10",
>   "spellcheck.extendedResults": "true",
>   "spellcheck.dictionary": [
> "dest_wordbreak",
> "dest_fuzzy"
>   ],
>   "q": "kosamui thailand",
>   "defType": "edismax",
>   "expand": "true",
>   "spellcheck.maxResultsForSuggest": "3",
>   "qf": "country_name region_name",
>   "spellcheck": "true",
>   "spellcheck.accuracy": "0.5",
>   "spellcheck.count": "20",
>   "spellcheck.collate": "true",
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-8807) NPE during spell checking when result collapsing is activated and index got more than one segment.

2017-03-21 Thread Matthias Krueger (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matthias Krueger updated SOLR-8807:
---
Attachment: SOLR-8807.patch

A simple solution would be to have CollapsingQParserPlugin behave like other 
DelegatingCollectors and call
{code}
super.doSetNextReader(context); 
{code}
in doSetNextReader. Not sure if omission was intentional though. This at least 
avoids the NPE we're seeing.

> NPE during spell checking when result collapsing is activated and index got 
> more than one segment.
> --
>
> Key: SOLR-8807
> URL: https://issues.apache.org/jira/browse/SOLR-8807
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 5.4
> Environment: Solr 5.4 with an index cosisting of two segments
>Reporter: Christian Danninger
>Priority: Critical
>  Labels: collapse, spellcheck
> Attachments: SOLR-8807.patch
>
>
> When using spellchecker with collapse/expand results, I got an NPE. Only 
> happend when the index consists of more than one segment. 
> {code}
> 11:30:33,505 WARN  [org.apache.solr.spelling.SpellCheckCollator] 
> (http-/0.0.0.0:8080-2) Exception trying to re-query to check if a spell check 
> possibility would return any hits.: java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:631)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:681)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:213)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:147)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:238)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:203)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:281)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> {code}
> {code}
>   
>   
> spellchecker
>   
> 
> {code}
> The query parameters are:
> {code}
>   "spellcheck.maxCollations": "5",
>   "q.op": "AND",
>   "fq": "{!collapse field=type}",
>   "spellcheck.maxCollationTries": "10",
>   "spellcheck.collateMaxCollectDocs": "10",
>   "spellcheck.alternativeTermCount": "10",
>   "spellcheck.extendedResults": "true",
>   "spellcheck.dictionary": [
> "dest_wordbreak",
> "dest_fuzzy"
>   ],
>   "q": "kosamui thailand",
>   "defType": "edismax",
>   "expand": "true",
>   "spellcheck.maxResultsForSuggest": "3",
>   "qf": "country_name region_name",
>   "spellcheck": "true",
>   "spellcheck.accuracy": "0.5",
>   "spellcheck.count": "20",
>   "spellcheck.collate": "true",
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-8807) NPE during spell checking when result collapsing is activated and index got more than one segment.

2017-03-21 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935520#comment-15935520
 ] 

Matthias Krueger commented on SOLR-8807:


This seems to be very much related to SOLR-9104 and SOLR-10336 or even the same 
cause.

> NPE during spell checking when result collapsing is activated and index got 
> more than one segment.
> --
>
> Key: SOLR-8807
> URL: https://issues.apache.org/jira/browse/SOLR-8807
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 5.4
> Environment: Solr 5.4 with an index cosisting of two segments
>Reporter: Christian Danninger
>Priority: Critical
>  Labels: collapse, spellcheck
>
> When using spellchecker with collapse/expand results, I got an NPE. Only 
> happend when the index consists of more than one segment. 
> {code}
> 11:30:33,505 WARN  [org.apache.solr.spelling.SpellCheckCollator] 
> (http-/0.0.0.0:8080-2) Exception trying to re-query to check if a spell check 
> possibility would return any hits.: java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:631)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:681)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:213)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:147)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:238)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:203)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:281)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> {code}
> {code}
>   
>   
> spellchecker
>   
> 
> {code}
> The query parameters are:
> {code}
>   "spellcheck.maxCollations": "5",
>   "q.op": "AND",
>   "fq": "{!collapse field=type}",
>   "spellcheck.maxCollationTries": "10",
>   "spellcheck.collateMaxCollectDocs": "10",
>   "spellcheck.alternativeTermCount": "10",
>   "spellcheck.extendedResults": "true",
>   "spellcheck.dictionary": [
> "dest_wordbreak",
> "dest_fuzzy"
>   ],
>   "q": "kosamui thailand",
>   "defType": "edismax",
>   "expand": "true",
>   "spellcheck.maxResultsForSuggest": "3",
>   "qf": "country_name region_name",
>   "spellcheck": "true",
>   "spellcheck.accuracy": "0.5",
>   "spellcheck.count": "20",
>   "spellcheck.collate": "true",
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-8807) NPE during spell checking when result collapsing is activated and index got more than one segment.

2017-03-21 Thread Matthias Krueger (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-8807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935514#comment-15935514
 ] 

Matthias Krueger commented on SOLR-8807:


My client is affected by this NPE, too. Some oddities of the SpellCheckCollator 
seem to misalign with behaviour of the CollapsingQParserPlugin's collectors.
* The {{checkResponse}} ResponseBuilder in SpellCheckCollator#collate is being 
built using the original params (including the fq=\{!collapse…\} param) *and* 
the original filters.
* The {{checkResponse}} ends up having the 
CollapsingQParserPlugin$CollapsingPostFilter twice in the {{filters}} list.
* They will end up in a DelegatingCollector chain 
CollapsingQParserPlugin$OrdScoreCollector -> 
CollapsingQParserPlugin$OrdScoreCollector.
* Unfortunately, this does not work well when there are documents returned from 
less than all segments. The two OrdScoreCollectors are instantiated with the 
number of segments and open an empty array of LeafReaderContexts but their 
values are only populated for the first OrdScoreCollector in 
IndexSearcher#search (via Collector#getLeafCollector -> 
OrdScoreCollector#doSetNextReader) calls.
* OrdScoreCollector (and the other collectors in CollapsingQParserPlugin) only 
passes the LeafReaderContexts to the delegate within finish() (not within 
#doSetNextReader) but stops when it deems all necessary segments consumed. When 
the delegate itself then performs the iteration it will NPE in
{code}
nextDocBase = currentContext+1 < contexts.length ? 
contexts[currentContext+1].docBase : maxDoc;
{code}


> NPE during spell checking when result collapsing is activated and index got 
> more than one segment.
> --
>
> Key: SOLR-8807
> URL: https://issues.apache.org/jira/browse/SOLR-8807
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 5.4
> Environment: Solr 5.4 with an index cosisting of two segments
>Reporter: Christian Danninger
>Priority: Critical
>  Labels: collapse, spellcheck
>
> When using spellchecker with collapse/expand results, I got an NPE. Only 
> happend when the index consists of more than one segment. 
> {code}
> 11:30:33,505 WARN  [org.apache.solr.spelling.SpellCheckCollator] 
> (http-/0.0.0.0:8080-2) Exception trying to re-query to check if a spell check 
> possibility would return any hits.: java.lang.NullPointerException
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:631)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.CollapsingQParserPlugin$OrdScoreCollector.finish(CollapsingQParserPlugin.java:681)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:213)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1672)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1491)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:557) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:525)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.spelling.SpellCheckCollator.collate(SpellCheckCollator.java:147)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.addCollationsToResponse(SpellCheckComponent.java:238)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:203)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:281)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
>  [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073) 
> [solr-core-5.4.0.jar:5.4.0 1718046 - upayavira - 2015-12-04 23:16:46]
> {code}
> {code}
>   
>   
> spellche