well, to sum it up... it doesn't really matter if I use standard or
dismax, at the moment both give me NullPointers for the same query,
although I didn't change anything since it was working ... it seems
totally random, sometimes it works a couple of times, sometimes it
doesn't :(
Weird...
Stefan
Am 22.09.2010 19:58, schrieb Stefan Moises:
Hi all,
wow, this is weird...
now before I file the JIRA issue - one thing I forgot to mention is
that I am using the edismax query parser.
I've just done the following:
1) searched with edismax parser:
/select?indent=on&version=2.2&q=beffy&fq=&start=0&rows=10&fl=*%2Cscore&qt=dismax&wt=standard&debugQuery=on&explainOther=&hl.fl=&spellcheck=true
and got the NullPointer as usual
2) changed "dismax" to "standard", just to make sure - using the same
query:
/select?indent=on&version=2.2&q=beffy&fq=&start=0&rows=10&fl=*%2Cscore&qt=standard&wt=standard&debugQuery=on&explainOther=&hl.fl=&spellcheck=true
and got 6 results
3) changed back to "dismax" and got the 6 results, again - no more
NullPointer... !?
Is that something cache-related maybe? Because I've had other indexes
which at first gave me the NullPointer exception using this setup and
after a while the synonyms were working, even with spellchecking
turned on and using edismax...
Just for reference, here is my handler config:
<requestHandler name="dismax" class="solr.SearchHandler" >
<lst name="defaults">
<str name="defType">edismax</str>
<str name="echoParams">explicit</str>
<float name="tie">0.05</float>
<str name="qf">
oxtitle^136.9 oxartnum^20.9 oxartnum_exact^30 oxartnum_rev^30
oxtags^20 seokeywords^25 seodesc^15 oxlongdesc^10 oxcattitle^80.4
allcattitles_exact^55 allcattitles^50 allcattitles_rev^45
allcattitles_preserve_rev^50 oxmanu_title^60.0 oxvendor_title^60.0
</str>
<str name="pf">
oxtitle^136.9 oxartnum^20.9 oxartnum_exact^30 oxartnum_rev^30
oxtags^20 seokeywords^25 seodesc^15 oxlongdesc^10 oxcattitle^80.4
allcattitles_exact^55 allcattitles^50 allcattitles_rev^45
allcattitles_preserve_rev^50 oxmanu_title^60.0 oxvendor_title^60.0
</str>
<str name="mm">
2<-1 5<-2 6<90%
</str>
<int name="ps">100</int>
<str name="q.alt">*:*</str>
<!-- example highlighter config, enable per-query with hl=true -->
<str name="hl.fl">oxcattitle oxtitle oxshortdesc oxlongdesc</str>
<!-- for this field, we want no fragmenting, just highlighting -->
<str name="f.name.hl.fragsize">0</str>
<!-- instructs Solr to return the field itself if no query terms are
found -->
<str name="f.name.hl.alternateField">oxtitle</str>
<str name="f.text.hl.fragmenter">regex</str> <!-- defined below -->
</lst>
<arr name="last-components">
<str>spellcheck</str>
<str>elevator</str>
</arr>
</requestHandler>
Thanks for any hint :)
Stefan
Am 21.09.2010 21:14, schrieb Stefan Moises:
Sure, no problem, I'll submit a JIRA entry :)
Am 21.09.2010 16:13, schrieb Robert Muir:
I don't think you should get an error like this from
SynonymFilter... would
you mind opening a JIRA issue?
On Tue, Sep 21, 2010 at 9:49 AM, Stefan Moises<moi...@shoptimax.de>
wrote:
Hi again,
well it turns out that it still doesn't work ...
Sometimes it works (i.e. for some cores), sometimes I still get the
nullpointer - e.g. if I create a new core and use the same settings
as a
working one, but index different data, then I add a synonym (e.g.
"foo =>
bar") and activate spellchecking, then search for "foo" - boom :(
And I can't really tell where this error comes from... any idea
where to
start looking?
Thanks,
Stefan
Am 01.09.2010 17:20, schrieb Stefan Moises:
doh, looks like I only forgot to add the spellcheck component to my
edismax request handler... now it works with:
...
<arr name="last-components">
<str>spellcheck</str>
<str>elevator</str>
</arr>
What's strange is that spellchecking seemed to work *without* that
entry,
too....
Cheers,
Stefan
Am 01.09.2010 13:33, schrieb Stefan Moises:
Hi there,
I am using Solr from SVN,
https://svn.apache.org/repos/asf/lucene/dev/trunk (my last
update/build
on my dev server was in July I think)...
I've encountered a strange problem when using the Spellcheck
component in
combination with the SynonymFilter...
My "text" field is pretty standard, using the default
synonyms.txt file:
<fieldType name="text" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<!-- in this example, we will only use synonyms at query time
<filter class="solr.SynonymFilterFactory"
synonyms="index_synonyms.txt"
ignoreCase="true" expand="false"/>
-->
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1"
generateNumberParts="1" catenateWords="1" catenateNumbers="1"
catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<!--<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/> -->
<filter class="solr.SnowballPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
ignoreCase="true" expand="true"/>
<filter class="solr.StopFilterFactory" ignoreCase="true"
words="stopwords.txt"/>
<filter class="solr.WordDelimiterFilterFactory"
generateWordParts="1"
generateNumberParts="1" catenateWords="0" catenateNumbers="0"
catenateAll="0" splitOnCaseChange="1"/>
<filter class="solr.LowerCaseFilterFactory"/>
<!--<filter class="solr.EnglishPorterFilterFactory"
protected="protwords.txt"/> -->
<filter class="solr.SnowballPorterFilterFactory"
protected="protwords.txt"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>
I have only added some terms at the end of synonyms.txt:
...
# Synonym mappings can be used for spelling correction too
pixima => pixma
tekanne => teekanne
teekane => teekanne
flashen => flaschen
flasen => flaschen
Here is my query and the exception... if I turn off spellcheck,
everything works as expected and the synonyms are found...
INFO: [db] webapp=/solr path=/select
params={mlt.minwl=3&spellcheck=true&facet=true&mlt.fl=oxmanu_oxid,oxvendor_oxid,oxtags,oxsearchkeys&spellcheck.q=flasen&mlt.mintf=1&facet.limit=-1&mlt=true&
json.nl=map&hl.fl=oxtitle&hl.fl=oxshortdesc&hl.fl=oxlongdesc&hl.fl=oxtags&hl.fl=seodesc&hl.fl=seokeywords&wt=json&hl=true&rows=10&version=1.2&mlt.mindf=1&debugQuery=true&facet.sort=lex&start=0&q=flasen&facet.field=oxcat_oxid&facet.field=oxcat_oxidtitle&facet.field=oxprice&facet.field=oxmanu_oxid&facet.field=oxmanu_oxidtitle&facet.field=oxvendor_oxid&facet.field=oxvendor_oxidtitle&facet.field=attrgroup_oxid&facet.field=attrgroup_oxidtitle&facet.field=attrgroup_oxidvalue&facet.field=attrvalue_oxid&facet.field=attrvalue_oxidtitle&facet.field=attr2attrgroup_oxidtitle&qt=dismax&spellcheck.build=false}
hits=2 status=500 QTime=14
01.09.2010 12:54:47 org.apache.solr.common.SolrException log
SCHWERWIEGEND: java.lang.NullPointerException
at
org.apache.lucene.util.AttributeSource.cloneAttributes(AttributeSource.java:470)
at
org.apache.lucene.analysis.synonym.SynonymFilter.incrementToken(SynonymFilter.java:128)
at
org.apache.lucene.analysis.core.StopFilter.incrementToken(StopFilter.java:260)
at
org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter.incrementToken(WordDelimiterFilter.java:336)
at
org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken(LowerCaseFilter.java:62)
at
org.apache.solr.handler.component.SpellCheckComponent.getTokens(SpellCheckComponent.java:380)
at
org.apache.solr.handler.component.SpellCheckComponent.process(SpellCheckComponent.java:127)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:203)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1323)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:465)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:619)
Thanks for any hint what I may be doing wrong! :)
Stefan
--
*******************************************
Stefan Moises
Senior Softwareentwickler
shoptimax GmbH
Guntherstraße 45 a
90461 Nürnberg
Amtsgericht Nürnberg HRB 21703
GF Friedrich Schreieck
Tel.: 0911/25566-25
Fax: 0911/25566-29
moi...@shoptimax.de
http://www.shoptimax.de
*******************************************
--
*******************************************
Stefan Moises
Senior Softwareentwickler
shoptimax GmbH
Guntherstraße 45 a
90461 Nürnberg
Amtsgericht Nürnberg HRB 21703
GF Friedrich Schreieck
Tel.: 0911/25566-25
Fax: 0911/25566-29
moi...@shoptimax.de
http://www.shoptimax.de
*******************************************
