Re: Solr suggester : duplicate suggestions
@Paras Lehana.. Thanks for the reply
Yes "and" is present in the stop words list.
Dhanesh S.RSenior Technical Leade : dhan...@hifx.co.in | w : www.hifx.in712
t : (+91) 484 4011750
m : (+91) 994 666 6703
On Fri, Jan 10, 2020 at 3:07 PM Paras Lehana
wrote:
> Hi Dhanesh,
>
> Although I handle Auto-Suggest, I have worked a little with Suggester
> component. Suggester provides results as you type. Do you really need it?
>
> Also, I don't know if I'm correct, but where have you described '&' to be
> replaced with 'and'? Is 'and' present in your stopwords list?
>
> I think posting the query and results for both cases of first problem will
> help us more.
>
> On Thu, 9 Jan 2020 at 20:50, Dhanesh Radhakrishnan
> wrote:
>
> > Dear all,
> > I'm facing two issues with solr suggester component.
> >
> > *First *
> > If I typed "Fire and safety", I'll get the result. But If I type "Fire &
> > safety" suggester is not showing
> >
> > *Second*
> > I'm getting duplicate suggestions in suggester
> >
> > "suggest": {
> > "categorySuggester": {
> > "software": {
> > "numFound": 100,
> > "suggestions": [
> > {
> > "term": "Software And Web Development||6070",
> > "weight": 0,
> > "payload": ""
> > },
> > {
> > "term": "Software And Web Development||6070",
> > "weight": 0,
> > "payload": ""
> > },
> > {
> > "term": "Software And Web Development||6070",
> > "weight": 0,
> > "payload": ""
> > }
> >
> >
> >
> >
> > ]
> > }
> > }
> > }
> >
> >
> >
> > Here is my configuration
> >
> > In solrconfig.xml
> >
> >
> >
> >
> > categorySuggester
> > AnalyzingInfixLookupFactory
> > text_suggest
> > DocumentDictionaryFactory
> > categoryAutoComplete
> >categoryRank
> > false
> > false
> > /dictionary/category
> > true
> > false
> >
> >
> >
> >
> >
> > In schema.xml
> >
> > > stored="true" multiValued="true" />
> >
> >
> > > positionIncrementGap="100">
> >
> >
> > > generateWordParts="1" generateNumberParts="1" catenateWords="1"
> > catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
> >
> >
> > > words="stopwords.txt"/>
> > > ignoreCase="true" expand="true"
> > tokenizerFactory="solr.KeywordTokenizerFactory"/>
> >
> >
> >
> >
> >
> >
> >
> http://localhost:8983/solr/core-name/suggest?suggest=true&suggest.q=software&suggest.build=false&suggest.dictionary=categorySuggester&wt=json
> >
> > Please help
> >
> > Thanks & Regards,
> > dhanesh s r
> >
> >
> > Dhanesh S.RSenior Technical Leade : dhan...@hifx.co.in | w :
> > www.hifx.in712
> > t : (+91) 484 4011750
> > m : (+91) 994 666 6703
> >
> > --
> > IMPORTANT: This is an e-mail from HiFX IT Media Services Pvt. Ltd. Its
> > content are confidential to the intended recipient. If you are not the
> > intended recipient, be advised that you have received this e-mail in
> error
> > and that any use, dissemination, forwarding, printing or copying of this
> > e-mail is strictly prohibited. It may not be disclosed to or used by
> > anyone
> > other than its intended recipient, nor may it be copied in any way. If
> > received in error, please email a reply to the sender, then delete it
> from
>
Re: Solr suggester : duplicate suggestions
Hi Dhanesh,
Although I handle Auto-Suggest, I have worked a little with Suggester
component. Suggester provides results as you type. Do you really need it?
Also, I don't know if I'm correct, but where have you described '&' to be
replaced with 'and'? Is 'and' present in your stopwords list?
I think posting the query and results for both cases of first problem will
help us more.
On Thu, 9 Jan 2020 at 20:50, Dhanesh Radhakrishnan
wrote:
> Dear all,
> I'm facing two issues with solr suggester component.
>
> *First *
> If I typed "Fire and safety", I'll get the result. But If I type "Fire &
> safety" suggester is not showing
>
> *Second*
> I'm getting duplicate suggestions in suggester
>
> "suggest": {
> "categorySuggester": {
> "software": {
> "numFound": 100,
> "suggestions": [
> {
> "term": "Software And Web Development||6070",
> "weight": 0,
> "payload": ""
> },
> {
> "term": "Software And Web Development||6070",
> "weight": 0,
> "payload": ""
> },
> {
> "term": "Software And Web Development||6070",
> "weight": 0,
> "payload": ""
> }
>
>
>
>
> ]
> }
> }
> }
>
>
>
> Here is my configuration
>
> In solrconfig.xml
>
>
>
>
> categorySuggester
> AnalyzingInfixLookupFactory
> text_suggest
> DocumentDictionaryFactory
> categoryAutoComplete
>categoryRank
> false
> false
> /dictionary/category
> true
> false
>
>
>
>
>
> In schema.xml
>
> stored="true" multiValued="true" />
>
>
> positionIncrementGap="100">
>
>
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>
>
> words="stopwords.txt"/>
> ignoreCase="true" expand="true"
> tokenizerFactory="solr.KeywordTokenizerFactory"/>
>
>
>
>
>
>
> http://localhost:8983/solr/core-name/suggest?suggest=true&suggest.q=software&suggest.build=false&suggest.dictionary=categorySuggester&wt=json
>
> Please help
>
> Thanks & Regards,
> dhanesh s r
>
>
> Dhanesh S.RSenior Technical Leade : dhan...@hifx.co.in | w :
> www.hifx.in712
> t : (+91) 484 4011750
> m : (+91) 994 666 6703
>
> --
> IMPORTANT: This is an e-mail from HiFX IT Media Services Pvt. Ltd. Its
> content are confidential to the intended recipient. If you are not the
> intended recipient, be advised that you have received this e-mail in error
> and that any use, dissemination, forwarding, printing or copying of this
> e-mail is strictly prohibited. It may not be disclosed to or used by
> anyone
> other than its intended recipient, nor may it be copied in any way. If
> received in error, please email a reply to the sender, then delete it from
> your system.
>
> Although this e-mail has been scanned for viruses, HiFX
> cannot ultimately accept any responsibility for viruses and it is your
> responsibility to scan attachments (if any).
>
> Before you print this email
> or attachments, please consider the negative environmental impacts
> associated with printing.
>
--
--
Regards,
*Paras Lehana* [65871]
Development Engineer, Auto-Suggest,
IndiaMART Intermesh Ltd.
8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303
Mob.: +91-9560911996
Work: 01203916600 | Extn: *8173*
--
*
*
<https://www.facebook.com/IndiaMART/videos/578196442936091/>
Solr suggester : duplicate suggestions
Dear all,
I'm facing two issues with solr suggester component.
*First *
If I typed "Fire and safety", I'll get the result. But If I type "Fire &
safety" suggester is not showing
*Second*
I'm getting duplicate suggestions in suggester
"suggest": {
"categorySuggester": {
"software": {
"numFound": 100,
"suggestions": [
{
"term": "Software And Web Development||6070",
"weight": 0,
"payload": ""
},
{
"term": "Software And Web Development||6070",
"weight": 0,
"payload": ""
},
{
"term": "Software And Web Development||6070",
"weight": 0,
"payload": ""
}
]
}
}
}
Here is my configuration
In solrconfig.xml
categorySuggester
AnalyzingInfixLookupFactory
text_suggest
DocumentDictionaryFactory
categoryAutoComplete
categoryRank
false
false
/dictionary/category
true
false
In schema.xml
http://localhost:8983/solr/core-name/suggest?suggest=true&suggest.q=software&suggest.build=false&suggest.dictionary=categorySuggester&wt=json
Please help
Thanks & Regards,
dhanesh s r
Dhanesh S.RSenior Technical Leade : dhan...@hifx.co.in | w : www.hifx.in712
t : (+91) 484 4011750
m : (+91) 994 666 6703
--
IMPORTANT: This is an e-mail from HiFX IT Media Services Pvt. Ltd. Its
content are confidential to the intended recipient. If you are not the
intended recipient, be advised that you have received this e-mail in error
and that any use, dissemination, forwarding, printing or copying of this
e-mail is strictly prohibited. It may not be disclosed to or used by anyone
other than its intended recipient, nor may it be copied in any way. If
received in error, please email a reply to the sender, then delete it from
your system.
Although this e-mail has been scanned for viruses, HiFX
cannot ultimately accept any responsibility for viruses and it is your
responsibility to scan attachments (if any).
Before you print this email
or attachments, please consider the negative environmental impacts
associated with printing.
Re: Error with Solr Suggester using lookupIml = FreeTextLookupFactory
This setting is pretty dangerous. It’ll build the suggester every time the Solr
instance starts. The DocumentDictionaryFactory will read _every_ document in
your index to extract the stored “suggest” field and create the dictionary.
But this points to the fact that you hadn’t built the dictionary when you
originally posted the problem, you can do this by issuing a command (curl it
in, or on the browser line) like:
blahblahblahblah/solr/collection/suggest?suggest.build=true&suggest=true
Best,
Erick
> On Nov 6, 2019, at 4:32 PM, Tyrone Tse wrote:
>
> It's working now that I changed the solrconfig.xml to
>
>
>
>mySuggester
>FreeTextLookupFactory
>DocumentDictionaryFactory
>suggest
>3
>
> name="suggestFreeTextAnalyzerFieldType">text_en_splitting
>*true*
>
>
>
> On Wed, Nov 6, 2019 at 3:11 PM Tyrone Tse wrote:
>
>> What's the command to build it
>>
>>
>> On Wed, Nov 6, 2019 at 3:06 PM Mikhail Khludnev wrote:
>>
>>> Hello,
>>>
>>> Have you build suggester before requesting?
>>>
>>> On Wed, Nov 6, 2019 at 12:50 PM Tyrone Tse wrote:
>>>
Solr version 8.1.1
My schema
>>> multiValued="false" indexed="true"/>
solconfig.xml
mySuggester
FreeTextLookupFactory
DocumentDictionaryFactory
suggest
3
>>> name="suggestFreeTextAnalyzerFieldType">text_en_splitting
false
>>>startup="lazy" >
true
10
suggest
The suggest query
>>> http://localhost:8983/solr/catalog/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.q=gin
works on Red Hat Enterprise Linux 7.6
it returns
{
"responseHeader":{
"status":0,
"QTime":0},
"suggest":{"mySuggester":{
"gin":{
"numFound":10,
"suggestions":[{
"term":"gin",
"weight":13613207305387128,
"payload":""},
{
"term":"ginjo",
"weight":3986422076966947,
"payload":""},
...
But when I on my Mac with OS High Sierra
Generates the error
"Lookup not supported at this time"
"java.lang.IllegalStateException: Lookup not supported at this
>>> time\n\tat
>>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:428)\n\tat
>>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)\n\tat
>>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)\n\tat
>>> org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)\n\tat
>>> org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)\n\tat
>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)\n\tat
>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)\n\tat
>>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542)\n\tat
>>>
>>>
>>> --
>>> Sincerely yours
>>> Mikhail Khludnev
>>>
>>
Re: Error with Solr Suggester using lookupIml = FreeTextLookupFactory
It's working now that I changed the solrconfig.xml to
mySuggester
FreeTextLookupFactory
DocumentDictionaryFactory
suggest
3
text_en_splitting
*true*
On Wed, Nov 6, 2019 at 3:11 PM Tyrone Tse wrote:
> What's the command to build it
>
>
> On Wed, Nov 6, 2019 at 3:06 PM Mikhail Khludnev wrote:
>
>> Hello,
>>
>> Have you build suggester before requesting?
>>
>> On Wed, Nov 6, 2019 at 12:50 PM Tyrone Tse wrote:
>>
>> > Solr version 8.1.1
>> >
>> > My schema
>> >
>> > > > multiValued="false" indexed="true"/>
>> >
>> >
>> > solconfig.xml
>> >
>> >
>> >
>> > mySuggester
>> > FreeTextLookupFactory
>> > DocumentDictionaryFactory
>> > suggest
>> > 3
>> >
>> > > > name="suggestFreeTextAnalyzerFieldType">text_en_splitting
>> > false
>> >
>> >
>> >
>> > > > startup="lazy" >
>> >
>> > true
>> > 10
>> >
>> >
>> > suggest
>> >
>> >
>> >
>> > The suggest query
>> >
>> >
>> http://localhost:8983/solr/catalog/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.q=gin
>> >
>> > works on Red Hat Enterprise Linux 7.6
>> >
>> > it returns
>> >
>> > {
>> > "responseHeader":{
>> > "status":0,
>> > "QTime":0},
>> > "suggest":{"mySuggester":{
>> > "gin":{
>> > "numFound":10,
>> > "suggestions":[{
>> > "term":"gin",
>> > "weight":13613207305387128,
>> > "payload":""},
>> > {
>> > "term":"ginjo",
>> > "weight":3986422076966947,
>> > "payload":""},
>> > ...
>> >
>> > But when I on my Mac with OS High Sierra
>> > Generates the error
>> >
>> > "Lookup not supported at this time"
>> >
>> > "java.lang.IllegalStateException: Lookup not supported at this
>> time\n\tat
>> >
>> >
>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:428)\n\tat
>> >
>> >
>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)\n\tat
>> >
>> >
>> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)\n\tat
>> >
>> >
>> org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)\n\tat
>> >
>> >
>> org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)\n\tat
>> >
>> >
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)\n\tat
>> >
>> >
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
>> > org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)\n\tat
>> >
>> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756)\n\tat
>> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542)\n\tat
>> >
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>>
>
Re: Error with Solr Suggester using lookupIml = FreeTextLookupFactory
What's the command to build it
On Wed, Nov 6, 2019 at 3:06 PM Mikhail Khludnev wrote:
> Hello,
>
> Have you build suggester before requesting?
>
> On Wed, Nov 6, 2019 at 12:50 PM Tyrone Tse wrote:
>
> > Solr version 8.1.1
> >
> > My schema
> >
> > > multiValued="false" indexed="true"/>
> >
> >
> > solconfig.xml
> >
> >
> >
> > mySuggester
> > FreeTextLookupFactory
> > DocumentDictionaryFactory
> > suggest
> > 3
> >
> > > name="suggestFreeTextAnalyzerFieldType">text_en_splitting
> > false
> >
> >
> >
> > > startup="lazy" >
> >
> > true
> > 10
> >
> >
> > suggest
> >
> >
> >
> > The suggest query
> >
> >
> http://localhost:8983/solr/catalog/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.q=gin
> >
> > works on Red Hat Enterprise Linux 7.6
> >
> > it returns
> >
> > {
> > "responseHeader":{
> > "status":0,
> > "QTime":0},
> > "suggest":{"mySuggester":{
> > "gin":{
> > "numFound":10,
> > "suggestions":[{
> > "term":"gin",
> > "weight":13613207305387128,
> > "payload":""},
> > {
> > "term":"ginjo",
> > "weight":3986422076966947,
> > "payload":""},
> > ...
> >
> > But when I on my Mac with OS High Sierra
> > Generates the error
> >
> > "Lookup not supported at this time"
> >
> > "java.lang.IllegalStateException: Lookup not supported at this time\n\tat
> >
> >
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:428)\n\tat
> >
> >
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)\n\tat
> >
> >
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)\n\tat
> >
> >
> org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)\n\tat
> >
> >
> org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)\n\tat
> >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)\n\tat
> >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
> > org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542)\n\tat
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>
Re: Error with Solr Suggester using lookupIml = FreeTextLookupFactory
Hello,
Have you build suggester before requesting?
On Wed, Nov 6, 2019 at 12:50 PM Tyrone Tse wrote:
> Solr version 8.1.1
>
> My schema
>
> multiValued="false" indexed="true"/>
>
>
> solconfig.xml
>
>
>
> mySuggester
> FreeTextLookupFactory
> DocumentDictionaryFactory
> suggest
> 3
>
> name="suggestFreeTextAnalyzerFieldType">text_en_splitting
> false
>
>
>
> startup="lazy" >
>
> true
> 10
>
>
> suggest
>
>
>
> The suggest query
>
> http://localhost:8983/solr/catalog/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.q=gin
>
> works on Red Hat Enterprise Linux 7.6
>
> it returns
>
> {
> "responseHeader":{
> "status":0,
> "QTime":0},
> "suggest":{"mySuggester":{
> "gin":{
> "numFound":10,
> "suggestions":[{
> "term":"gin",
> "weight":13613207305387128,
> "payload":""},
> {
> "term":"ginjo",
> "weight":3986422076966947,
> "payload":""},
> ...
>
> But when I on my Mac with OS High Sierra
> Generates the error
>
> "Lookup not supported at this time"
>
> "java.lang.IllegalStateException: Lookup not supported at this time\n\tat
>
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:428)\n\tat
>
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)\n\tat
>
> org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)\n\tat
>
> org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)\n\tat
>
> org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)\n\tat
>
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542)\n\tat
>
--
Sincerely yours
Mikhail Khludnev
Error with Solr Suggester using lookupIml = FreeTextLookupFactory
Solr version 8.1.1
My schema
solconfig.xml
mySuggester
FreeTextLookupFactory
DocumentDictionaryFactory
suggest
3
text_en_splitting
false
true
10
suggest
The suggest query
http://localhost:8983/solr/catalog/suggest?suggest=true&suggest.dictionary=mySuggester&suggest.q=gin
works on Red Hat Enterprise Linux 7.6
it returns
{
"responseHeader":{
"status":0,
"QTime":0},
"suggest":{"mySuggester":{
"gin":{
"numFound":10,
"suggestions":[{
"term":"gin",
"weight":13613207305387128,
"payload":""},
{
"term":"ginjo",
"weight":3986422076966947,
"payload":""},
...
But when I on my Mac with OS High Sierra
Generates the error
"Lookup not supported at this time"
"java.lang.IllegalStateException: Lookup not supported at this time\n\tat
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:428)\n\tat
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:399)\n\tat
org.apache.lucene.search.suggest.analyzing.FreeTextSuggester.lookup(FreeTextSuggester.java:388)\n\tat
org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:243)\n\tat
org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:264)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:756)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:542)\n\tat
Behavior of a Solr suggester using spellchecker
Dear Solr community,
I am facing a behavior with a suggest request handler that does not look normal
to me, and I would like to understand.
* Here is the configuration of the suggest component :
suggest
org.apache.solr.spelling.suggest.Suggester
org.apache.solr.spelling.suggest.tst.TSTLookup
suggest
${autocomplete.threshold:0.005}
true
* Here is the configuration of the request handler :
suggest
true
suggest
true
10
score
true
true
10
100
10
true
suggest
query
{!manifoldCFSecurity}
For info, the "manifoldCFSecurity" parameter leads to a MCFQueryParser that
generates a boolean query to be applied to the search.
When one performs a request on this handler, the parse() method of the
MCFQueryParser is called by Solr for the main query, but also one additional
time per suggested term. This behavior looks weird to me as it appears to be
suboptimal to do one query per suggested term.
Is it normal ? Did I make a mistake in the configurations ?
Thanks for your help.
Regards,
Cedric
Re: Problem with solr suggester in case of non-ASCII characters
Hi Erick,
Thanks your advice.
I already removed it from the field definition used by the suggester and it
works great. I will consider to took it from the entire processing of the
other fields. I have only 7000 docs with index size of 18MB so far, so the
memory footprint is not a key issue for me.
Best,
Roland
Erick Erickson ezt írta (időpont: 2019. júl. 31.,
Sze, 14:24):
> Roland:
>
> Have you considered just not using stopwords anywhere? Largely they’re a
> holdover
> from a long time ago when every byte counted. Plus using stopwords has
> “interesting”
> issues with things like highlighting and phrase queries and the like.
>
> Sure, not using stopwords will make your index larger, but so will a
> copyfield…
>
> Your call of course, but stopwords are over-used IMO.
>
> I’m stealing Walter Underwood’s thunder here ;)
>
> Best,
> Erick
>
> > On Jul 30, 2019, at 2:11 PM, Szűcs Roland
> wrote:
> >
> > Hi Furkan,
> >
> > Thanks the suggestion, I always forget the most effective debugging tool
> > the analysis page.
> >
> > It turned out that "Jó" was a stop word and it was eliminated during the
> > text analysis. What I will do is to create a new field type but without
> > stop word removal and I will use it like this:
> > > name="suggestAnalyzerFieldType">short_text_hu_without_stop_removal
> >
> > Thanks again
> >
> > Roland
> >
> > Furkan KAMACI ezt írta (időpont: 2019. júl.
> 30.,
> > K, 16:17):
> >
> >> Hi Roland,
> >>
> >> Could you check Analysis tab (
> >> https://lucene.apache.org/solr/guide/8_1/analysis-screen.html) and tell
> >> how
> >> the term is analyzed for both query and index?
> >>
> >> Kind Regards,
> >> Furkan KAMACI
> >>
> >> On Tue, Jul 30, 2019 at 4:50 PM Szűcs Roland <
> szucs.rol...@bookandwalk.hu>
> >> wrote:
> >>
> >>> Hi All,
> >>>
> >>> I have an author suggester (searchcomponent and the related request
> >>> handler) defined in solrconfig:
> >>>
> >>>>
> >>>
> >>> author
> >>> AnalyzingInfixLookupFactory
> >>> DocumentDictionaryFactory
> >>> BOOK_productAuthor
> >>> short_text_hu
> >>> suggester_infix_author
> >>> false
> >>> false
> >>> 2
> >>>
> >>>
> >>>
> >>> >>> startup="lazy" >
> >>>
> >>> true
> >>> 10
> >>> author
> >>>
> >>>
> >>> suggest
> >>>
> >>>
> >>>
> >>> Author field has just a minimal text processing in query and index time
> >>> based on the following definition:
> >>> >>> positionIncrementGap="100" multiValued="true">
> >>>
> >>>
> >>>
> >>> >>> ignoreCase="true"/>
> >>>
> >>>
> >>>
> >>>
> >>> >>> ignoreCase="true"/>
> >>>
> >>>
> >>>
> >>> >>> docValues="true"/>
> >>> >>> docValues="true" multiValued="true"/>
> >>> >>> positionIncrementGap="100">
> >>>
> >>>
> >>>
> >>> >> words="lang/stopwords_ar.txt"
> >>> ignoreCase="true"/>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> When I use qeries with only ASCII characters, the results are correct:
> >>> "Al":{
> >>> "term":"Alexandre Dumas", "weight":0, "payload":""}
> >>>
> >>> When I try it with Hungarian authorname with special character:
> >>> "Jó":"author":{
> >>> "Jó":{ "numFound":0, "suggestions":[]}}
> >>>
> >>> When I try it with three letters, it works again:
> >>> "Józ":"author":{
> >>> "Józ":{ "numFound":10, "suggestions":[{ "term":"Bajza József", "
> >>> weight":0, "payload":""}, { "term":"Eötvös József", "weight":0,
> "
> >>> payload":""}, { "term":"Eötvös József", "weight":0,
> >> "payload":""}, {
> >>> "term":"Eötvös József", "weight":0, "payload":""}, {
> >>> "term":"József
> >>> Attila", "weight":0, "payload":""}..
> >>>
> >>> Any idea how can it happen that a longer string has more matches than a
> >>> shorter one. It is inconsistent. What can I do to fix it as it would
> >>> results poor customer experience.
> >>> They would feel that sometimes they need 2 sometimes 3 characters to
> get
> >>> suggestions.
> >>>
> >>> Thanks in advance,
> >>> Roland
> >>>
> >>
>
>
Re: Problem with solr suggester in case of non-ASCII characters
Roland:
Have you considered just not using stopwords anywhere? Largely they’re a
holdover
from a long time ago when every byte counted. Plus using stopwords has
“interesting”
issues with things like highlighting and phrase queries and the like.
Sure, not using stopwords will make your index larger, but so will a copyfield…
Your call of course, but stopwords are over-used IMO.
I’m stealing Walter Underwood’s thunder here ;)
Best,
Erick
> On Jul 30, 2019, at 2:11 PM, Szűcs Roland wrote:
>
> Hi Furkan,
>
> Thanks the suggestion, I always forget the most effective debugging tool
> the analysis page.
>
> It turned out that "Jó" was a stop word and it was eliminated during the
> text analysis. What I will do is to create a new field type but without
> stop word removal and I will use it like this:
> name="suggestAnalyzerFieldType">short_text_hu_without_stop_removal
>
> Thanks again
>
> Roland
>
> Furkan KAMACI ezt írta (időpont: 2019. júl. 30.,
> K, 16:17):
>
>> Hi Roland,
>>
>> Could you check Analysis tab (
>> https://lucene.apache.org/solr/guide/8_1/analysis-screen.html) and tell
>> how
>> the term is analyzed for both query and index?
>>
>> Kind Regards,
>> Furkan KAMACI
>>
>> On Tue, Jul 30, 2019 at 4:50 PM Szűcs Roland
>> wrote:
>>
>>> Hi All,
>>>
>>> I have an author suggester (searchcomponent and the related request
>>> handler) defined in solrconfig:
>>>
>>>>
>>>
>>> author
>>> AnalyzingInfixLookupFactory
>>> DocumentDictionaryFactory
>>> BOOK_productAuthor
>>> short_text_hu
>>> suggester_infix_author
>>> false
>>> false
>>> 2
>>>
>>>
>>>
>>> >> startup="lazy" >
>>>
>>> true
>>> 10
>>> author
>>>
>>>
>>> suggest
>>>
>>>
>>>
>>> Author field has just a minimal text processing in query and index time
>>> based on the following definition:
>>> >> positionIncrementGap="100" multiValued="true">
>>>
>>>
>>>
>>> >> ignoreCase="true"/>
>>>
>>>
>>>
>>>
>>> >> ignoreCase="true"/>
>>>
>>>
>>>
>>> >> docValues="true"/>
>>> >> docValues="true" multiValued="true"/>
>>> >> positionIncrementGap="100">
>>>
>>>
>>>
>>> > words="lang/stopwords_ar.txt"
>>> ignoreCase="true"/>
>>>
>>>
>>>
>>>
>>>
>>> When I use qeries with only ASCII characters, the results are correct:
>>> "Al":{
>>> "term":"Alexandre Dumas", "weight":0, "payload":""}
>>>
>>> When I try it with Hungarian authorname with special character:
>>> "Jó":"author":{
>>> "Jó":{ "numFound":0, "suggestions":[]}}
>>>
>>> When I try it with three letters, it works again:
>>> "Józ":"author":{
>>> "Józ":{ "numFound":10, "suggestions":[{ "term":"Bajza József", "
>>> weight":0, "payload":""}, { "term":"Eötvös József", "weight":0, "
>>> payload":""}, { "term":"Eötvös József", "weight":0,
>> "payload":""}, {
>>> "term":"Eötvös József", "weight":0, "payload":""}, {
>>> "term":"József
>>> Attila", "weight":0, "payload":""}..
>>>
>>> Any idea how can it happen that a longer string has more matches than a
>>> shorter one. It is inconsistent. What can I do to fix it as it would
>>> results poor customer experience.
>>> They would feel that sometimes they need 2 sometimes 3 characters to get
>>> suggestions.
>>>
>>> Thanks in advance,
>>> Roland
>>>
>>
Re: Problem with solr suggester in case of non-ASCII characters
Hi Furkan,
Thanks the suggestion, I always forget the most effective debugging tool
the analysis page.
It turned out that "Jó" was a stop word and it was eliminated during the
text analysis. What I will do is to create a new field type but without
stop word removal and I will use it like this:
short_text_hu_without_stop_removal
Thanks again
Roland
Furkan KAMACI ezt írta (időpont: 2019. júl. 30.,
K, 16:17):
> Hi Roland,
>
> Could you check Analysis tab (
> https://lucene.apache.org/solr/guide/8_1/analysis-screen.html) and tell
> how
> the term is analyzed for both query and index?
>
> Kind Regards,
> Furkan KAMACI
>
> On Tue, Jul 30, 2019 at 4:50 PM Szűcs Roland
> wrote:
>
> > Hi All,
> >
> > I have an author suggester (searchcomponent and the related request
> > handler) defined in solrconfig:
> >
> > >
> >
> > author
> > AnalyzingInfixLookupFactory
> > DocumentDictionaryFactory
> > BOOK_productAuthor
> > short_text_hu
> > suggester_infix_author
> > false
> > false
> > 2
> >
> >
> >
> > > startup="lazy" >
> >
> > true
> > 10
> > author
> >
> >
> > suggest
> >
> >
> >
> > Author field has just a minimal text processing in query and index time
> > based on the following definition:
> > > positionIncrementGap="100" multiValued="true">
> >
> >
> >
> >> ignoreCase="true"/>
> >
> >
> >
> >
> >> ignoreCase="true"/>
> >
> >
> >
> >> docValues="true"/>
> >> docValues="true" multiValued="true"/>
> >> positionIncrementGap="100">
> >
> >
> >
> >words="lang/stopwords_ar.txt"
> > ignoreCase="true"/>
> >
> >
> >
> >
> >
> > When I use qeries with only ASCII characters, the results are correct:
> > "Al":{
> > "term":"Alexandre Dumas", "weight":0, "payload":""}
> >
> > When I try it with Hungarian authorname with special character:
> > "Jó":"author":{
> > "Jó":{ "numFound":0, "suggestions":[]}}
> >
> > When I try it with three letters, it works again:
> > "Józ":"author":{
> > "Józ":{ "numFound":10, "suggestions":[{ "term":"Bajza József", "
> > weight":0, "payload":""}, { "term":"Eötvös József", "weight":0, "
> > payload":""}, { "term":"Eötvös József", "weight":0,
> "payload":""}, {
> > "term":"Eötvös József", "weight":0, "payload":""}, {
> > "term":"József
> > Attila", "weight":0, "payload":""}..
> >
> > Any idea how can it happen that a longer string has more matches than a
> > shorter one. It is inconsistent. What can I do to fix it as it would
> > results poor customer experience.
> > They would feel that sometimes they need 2 sometimes 3 characters to get
> > suggestions.
> >
> > Thanks in advance,
> > Roland
> >
>
Re: Problem with solr suggester in case of non-ASCII characters
Hi Roland,
Could you check Analysis tab (
https://lucene.apache.org/solr/guide/8_1/analysis-screen.html) and tell how
the term is analyzed for both query and index?
Kind Regards,
Furkan KAMACI
On Tue, Jul 30, 2019 at 4:50 PM Szűcs Roland
wrote:
> Hi All,
>
> I have an author suggester (searchcomponent and the related request
> handler) defined in solrconfig:
>
> >
>
> author
> AnalyzingInfixLookupFactory
> DocumentDictionaryFactory
> BOOK_productAuthor
> short_text_hu
> suggester_infix_author
> false
> false
> 2
>
>
>
> startup="lazy" >
>
> true
> 10
> author
>
>
> suggest
>
>
>
> Author field has just a minimal text processing in query and index time
> based on the following definition:
> positionIncrementGap="100" multiValued="true">
>
>
>
>ignoreCase="true"/>
>
>
>
>
>ignoreCase="true"/>
>
>
>
>docValues="true"/>
>docValues="true" multiValued="true"/>
>positionIncrementGap="100">
>
>
>
>ignoreCase="true"/>
>
>
>
>
>
> When I use qeries with only ASCII characters, the results are correct:
> "Al":{
> "term":"Alexandre Dumas", "weight":0, "payload":""}
>
> When I try it with Hungarian authorname with special character:
> "Jó":"author":{
> "Jó":{ "numFound":0, "suggestions":[]}}
>
> When I try it with three letters, it works again:
> "Józ":"author":{
> "Józ":{ "numFound":10, "suggestions":[{ "term":"Bajza József", "
> weight":0, "payload":""}, { "term":"Eötvös József", "weight":0, "
> payload":""}, { "term":"Eötvös József", "weight":0, "payload":""}, {
> "term":"Eötvös József", "weight":0, "payload":""}, {
> "term":"József
> Attila", "weight":0, "payload":""}..
>
> Any idea how can it happen that a longer string has more matches than a
> shorter one. It is inconsistent. What can I do to fix it as it would
> results poor customer experience.
> They would feel that sometimes they need 2 sometimes 3 characters to get
> suggestions.
>
> Thanks in advance,
> Roland
>
Problem with solr suggester in case of non-ASCII characters
Hi All,
I have an author suggester (searchcomponent and the related request
handler) defined in solrconfig:
>
author
AnalyzingInfixLookupFactory
DocumentDictionaryFactory
BOOK_productAuthor
short_text_hu
suggester_infix_author
false
false
2
true
10
author
suggest
Author field has just a minimal text processing in query and index time
based on the following definition:
When I use qeries with only ASCII characters, the results are correct:
"Al":{
"term":"Alexandre Dumas", "weight":0, "payload":""}
When I try it with Hungarian authorname with special character:
"Jó":"author":{
"Jó":{ "numFound":0, "suggestions":[]}}
When I try it with three letters, it works again:
"Józ":"author":{
"Józ":{ "numFound":10, "suggestions":[{ "term":"Bajza József", "
weight":0, "payload":""}, { "term":"Eötvös József", "weight":0, "
payload":""}, { "term":"Eötvös József", "weight":0, "payload":""}, {
"term":"Eötvös József", "weight":0, "payload":""}, {
"term":"József
Attila", "weight":0, "payload":""}..
Any idea how can it happen that a longer string has more matches than a
shorter one. It is inconsistent. What can I do to fix it as it would
results poor customer experience.
They would feel that sometimes they need 2 sometimes 3 characters to get
suggestions.
Thanks in advance,
Roland
Re: SOLR Suggester returns either the full field value or single terms only
On Wed, Jun 19, 2019 at 12:20:43PM -0700, ppunet wrote: > As the SuggeterComponent provides the 'entire content' of the field in the > suggestions. How is it possible to have Suggester to return only part of the > content of the field, instead of the entire content, which in my scenario > quite long? Possibly worthless newbie suggestion: could you use highlighting to locate the text that triggered the suggestion, and just chop off leading and trailing context down to a reasonable length surrounding the match? Kind of like you'd see in a printed KWIC index: give as much context as will fit the available space, and don't worry about the rest. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu signature.asc Description: PGP signature
Re: SOLR Suggester returns either the full field value or single terms only
Hey, As the SuggeterComponent provides the 'entire content' of the field in the suggestions. How is it possible to have Suggester to return only part of the content of the field, instead of the entire content, which in my scenario quite long? -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr Suggester component not returning any results
Hi, Will you be able to post your configurations for your /suggest requestHandler in solrconfig.xml? Regards, Edwin On Wed, 20 Mar 2019 at 02:40, Deoxyribonucleic_DNA ... < deoxyribonucleic_...@hotmail.com> wrote: > Hi, I'm trying to implement the suggester component in Solr, based off > the tutorial: > > > https://lucene.apache.org/solr/guide/7_7/suggester.html#suggester-search-component-parameters > > I'm not getting any errors about how the suggester is set up or anything, > but when I try to use it I get 0 results returned. Including images of the > schema/config parts I added to my files, as shown to do in the tutorial, > can someone take a look? Or does anyone have any suggestions based on prior > experience or common problems? > > Also, I've got solr 7.6. There wasn't some change that would affect this > just in that slight version change was there? I > > Thanks in advance. > > > > > > > Link used in search: > > > http://localhost:8984/solr/nutch/suggest?suggest=true&; > suggest.build=true&suggest.dictionary=mySuggester&suggest.q=Canad > > (due to documents containing "Canada" as the intended word) > > Result: > > >
Solr Suggester component not returning any results
Hi, I'm trying to implement the suggester component in Solr, based off the tutorial: https://lucene.apache.org/solr/guide/7_7/suggester.html#suggester-search-component-parameters I'm not getting any errors about how the suggester is set up or anything, but when I try to use it I get 0 results returned. Including images of the schema/config parts I added to my files, as shown to do in the tutorial, can someone take a look? Or does anyone have any suggestions based on prior experience or common problems? Also, I've got solr 7.6. There wasn't some change that would affect this just in that slight version change was there? I Thanks in advance. [cid:0e5daa48-57aa-439d-a227-8ad4d2249e18] [cid:f73bacaf-e3b4-4767-b42a-55d53678b623] Link used in search: http://localhost:8984/solr/nutch/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&suggest.q=Canad (due to documents containing "Canada" as the intended word) Result: [cid:d8bec75d-1d45-4b5f-8dc7-3ffaf22ee7a7]
Re: Scores with Solr Suggester
Hi Christine, it depends on the suggester implementation, the one that got closer in having a score implementation is the BlendedInfix[1] but it is still in the TO DO phase. Feel free to contribute it if you like ! [1] https://sease.io/2018/06/apache-lucene-blendedinfixsuggester-how-it-works-bugs-and-improvements.html - --- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Scores with Solr Suggester
Hi Christine, suggesters work differently than regular search as they complete an input query, usually based on a state machine built from a dictionary. If you want the similarity of input and suggestion, you can create a search component to compute it yourself and set the value in the payload field. If suggestions should be returned in a re-ordered way already, it would have to be done in such a search component as well. Cheers, Chris On Mon, Jul 2, 2018 at 8:31 PM, Buckler, Christine < christine.buck...@nordstrom.com> wrote: > Is it possible to return a score field for Suggester results like it does > with standard search? I am looking for the score which quantifies how close > of a match between type entered and suggestion result (not the weight > associated with the suggestion). Is this possible? > > > > Christine Buckler > > [image: id:image001.png@01D3F81F.AF489300]christinebuckler > > [image: id:image002.png@01D3F81F.AF489300]206.295.6772 >
Scores with Solr Suggester
Is it possible to return a score field for Suggester results like it does with standard search? I am looking for the score which quantifies how close of a match between type entered and suggestion result (not the weight associated with the suggestion). Is this possible? Christine Buckler [id:image001.png@01D3F81F.AF489300]christinebuckler [id:image002.png@01D3F81F.AF489300]206.295.6772
Solr suggester build takes a very long time
So when following command is run to build solr suggester: ?suggest.build=true It takes a very long time to finish. I found out that this is because each time dictionary is built, it does not build delta, it rebuilds the entire dictionary. Is there a way to speed up the suggester build time? TIA -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solr suggester query with quotes produces different results
Hi guys, I have the Suggester configured using the FreeTextFactory. Noticed that if I dont use quotation marks, I only get single term results. If i use quotation marks around my query, then I only get results that are comprised of multiple terms. There is no configuration that would return both types of results with a single query. Thanks Angel
Re: SOLR Suggester returns either the full field value or single terms only
Hi Alessandro,
Thanks. I've experimented a bit more and here is what I have discovered -
If my query is enclosed with quotes, then i get multi terms, if it is not
enclosed in quotes, i only get single terms,
Example: will only return single terms:
http://localhost:8080/solr//suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=
*video*
Sample result:
{"responseHeader":{"status":0,"QTime":32},"suggest":{"mySuggester":{"video":{"numFound":10,"suggestions":[{"term":"video","weight":4169589427484439,"payload":""},{"term":"videos","weight":274540867653296,"payload":""},{"term":"videopilot5011pilot2016","weight":137270433826648,"payload":""},{"term":"videoplaylisthttp","weight":137270433826648,"payload":""},{"term":"videotransition","weight":34317608456662,"payload":""}]
Example: multiterm results
http://localhost:8080/solr//suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=
*"video"*
{"responseHeader":{"status":0,"QTime":12},"suggest":{"mySuggester":{"\"video\"":{"numFound":10,"suggestions":[{"term":"video
shows","weight":3491976244405923328,"payload":""},{"term":"video
from","weight":948906588153783552,"payload":""},{"term":"video
leaving","weight":189781317630756704,"payload":""},{"term":"video
entitled","weight":151825054104605376,"payload":""}}]
What's more , if my query is something like "video g", i get results that
don't include "video", for example:
http://localhost:8080/solr//suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=*"video
g"*
{"responseHeader":{"status":0,"QTime":48},"suggest":{"mySuggester":{"\"video
g\"":{"numFound":10,"suggestions":[{"term":"g
eo","weight":952090016707589760,"payload":""},{"term":"g
em","weight":297528130221121792,"payload":""},{"term":"g
spokesperson","weight":297528130221121792,"payload":""},{"term":"g
prepares","weight":238022504176897440,"payload":""}]
and so on. I am not sure why it's working in this way, but it doesn't seem
right to me. I am running on SOLR 5.1, if it makes any difference. Could it
be the solr version?
Thanks again,
Angel
On Tue, Jun 27, 2017 at 11:43 AM, alessandro.benedetti wrote:
> Hi Angel,
> can you give me an example of query, a couple of documents of example, and
> the suggestions you get ( which you don't expect) ?
>
> The config seems fine ( I remember there were some tricky problems with the
> default separator, but a space should be fine there).
>
> Cheers
>
>
>
> -
> ---
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/SOLR-Suggester-returns-either-the-full-field-
> value-or-single-terms-only-tp4342763p4342987.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: SOLR Suggester returns either the full field value or single terms only
Hi Angel, can you give me an example of query, a couple of documents of example, and the suggestions you get ( which you don't expect) ? The config seems fine ( I remember there were some tricky problems with the default separator, but a space should be fine there). Cheers - --- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-Suggester-returns-either-the-full-field-value-or-single-terms-only-tp4342763p4342987.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Suggester returns either the full field value or single terms only
hi Alessandro, Thanks very much. I've read your article - which is really great by the way! , and configured my suggester in the following way: mySuggester FreeTextLookupFactory DocumentDictionaryFactory content_suggest true 3 text_general The issue is that i am still getting single term suggestions. Sometimes i am getting multiple terms but they are not separated by a space, but by other characters like _ and :, for example "breaking_news" or "breaking:news", whenever i type "brea" or "br". Do you see anything wrong with my setup ? Thank you Angel On Mon, Jun 26, 2017 at 6:04 PM, govind nitk wrote: > Hi Alessandro, > > Thanks for clarification. > > > > On Mon, Jun 26, 2017 at 4:53 PM, alessandro.benedetti < > a.benede...@sease.io> > wrote: > > > " Don't use an heavy Analyzers, the suggested terms will come from the > > index, > > so be sure they are meaningful tokens. A really basic analyser is > > suggested, > > stop words and stemming are not " > > > > This means that your suggestions will come from the index, so if you use > > heavy analysers you can get terms suggested which are not really useful : > > > > e.g. > > > > Solr is an amazing search engine > > > > If you have some stemmer in your analysis chain, you will have this > > behavior > > : > > > > q= ama > > result : amaz search engin > > > > So it is better to have this lookup strategy configured on top of a light > > analysed field ( or copyfield). > > > > > > > > > > > > - > > --- > > Alessandro Benedetti > > Search Consultant, R&D Software Engineer, Director > > Sease Ltd. - www.sease.io > > -- > > View this message in context: http://lucene.472066.n3. > > nabble.com/SOLR-Suggester-returns-either-the-full-field- > > value-or-single-terms-only-tp4342763p4342807.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > >
Re: SOLR Suggester returns either the full field value or single terms only
Hi Alessandro, Thanks for clarification. On Mon, Jun 26, 2017 at 4:53 PM, alessandro.benedetti wrote: > " Don't use an heavy Analyzers, the suggested terms will come from the > index, > so be sure they are meaningful tokens. A really basic analyser is > suggested, > stop words and stemming are not " > > This means that your suggestions will come from the index, so if you use > heavy analysers you can get terms suggested which are not really useful : > > e.g. > > Solr is an amazing search engine > > If you have some stemmer in your analysis chain, you will have this > behavior > : > > q= ama > result : amaz search engin > > So it is better to have this lookup strategy configured on top of a light > analysed field ( or copyfield). > > > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > View this message in context: http://lucene.472066.n3. > nabble.com/SOLR-Suggester-returns-either-the-full-field- > value-or-single-terms-only-tp4342763p4342807.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: SOLR Suggester returns either the full field value or single terms only
" Don't use an heavy Analyzers, the suggested terms will come from the index, so be sure they are meaningful tokens. A really basic analyser is suggested, stop words and stemming are not " This means that your suggestions will come from the index, so if you use heavy analysers you can get terms suggested which are not really useful : e.g. Solr is an amazing search engine If you have some stemmer in your analysis chain, you will have this behavior : q= ama result : amaz search engin So it is better to have this lookup strategy configured on top of a light analysed field ( or copyfield). - --- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-Suggester-returns-either-the-full-field-value-or-single-terms-only-tp4342763p4342807.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Suggester returns either the full field value or single terms only
Hi alessandro, Really nice article. Can you brief us on "*Don't use an heavy Analyzers*" ? Regards, Govind On Mon, Jun 26, 2017 at 2:19 PM, alessandro.benedetti wrote: > Hi Angel, > your are looking for the Free Text lookup approach. > You find more info in [1] and [2] > > [1] > https://lucene.apache.org/solr/guide/6_6/suggester.html#Suggester- > FreeTextLookupFactory > [2] http://alexbenedetti.blogspot.co.uk/2015/07/solr-you-complete-me.html > > > > - > --- > Alessandro Benedetti > Search Consultant, R&D Software Engineer, Director > Sease Ltd. - www.sease.io > -- > View this message in context: http://lucene.472066.n3. > nabble.com/SOLR-Suggester-returns-either-the-full-field- > value-or-single-terms-only-tp4342763p4342790.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Re: SOLR Suggester returns either the full field value or single terms only
Hi Angel, your are looking for the Free Text lookup approach. You find more info in [1] and [2] [1] https://lucene.apache.org/solr/guide/6_6/suggester.html#Suggester-FreeTextLookupFactory [2] http://alexbenedetti.blogspot.co.uk/2015/07/solr-you-complete-me.html - --- Alessandro Benedetti Search Consultant, R&D Software Engineer, Director Sease Ltd. - www.sease.io -- View this message in context: http://lucene.472066.n3.nabble.com/SOLR-Suggester-returns-either-the-full-field-value-or-single-terms-only-tp4342763p4342790.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Suggester returns either the full field value or single terms only
Hi Angel, Please Look at these documents. 1. https://home.apache.org/~ctargett/RefGuidePOC/jekyll-full/suggester.html 2. https://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.ShingleFilterFactory Regards, Govind On Mon, Jun 26, 2017 at 3:12 AM, Angel Todorov wrote: > Hi guys, > > I am trying to configure the Suggester in a way that i get google-style > auto suggestions: > > - I don't want the suggestions to be the _whole_ field value > - I don't want the suggestions to be single terms > > For example, if I have a field that has the value "The brown fox jumped > over the fence" > > and I type "br" for example, I would like the get things like "Brown" and > "brown fox", but not the whole sentence. Also, i don't want my results to > be just single terms, but to also include phrases. > > I have tried a lot of configurations and have found out that if I use the > Document Dictionary Factory, I get the whole field value as a result, If I > use the Fuzzy Dictionary, I only get single terms as results. Nothing > similar to my requirements. My config is properly configured and my field > type is stored, because i am getting results, it's just that the results > are not what I'd expect. > > Would greatly appreciate if you can guide me to the right config. > > Thanks, > Angel >
SOLR Suggester returns either the full field value or single terms only
Hi guys, I am trying to configure the Suggester in a way that i get google-style auto suggestions: - I don't want the suggestions to be the _whole_ field value - I don't want the suggestions to be single terms For example, if I have a field that has the value "The brown fox jumped over the fence" and I type "br" for example, I would like the get things like "Brown" and "brown fox", but not the whole sentence. Also, i don't want my results to be just single terms, but to also include phrases. I have tried a lot of configurations and have found out that if I use the Document Dictionary Factory, I get the whole field value as a result, If I use the Fuzzy Dictionary, I only get single terms as results. Nothing similar to my requirements. My config is properly configured and my field type is stored, because i am getting results, it's just that the results are not what I'd expect. Would greatly appreciate if you can guide me to the right config. Thanks, Angel
Re: Solr Suggester
For the suggester, the field must be indexed. It's not necessary to have it stored. Best, Michael Am 22.12.2016 um 11:24 schrieb Furkan KAMACI: > Hi Emir, > > As far as I know, it should be enough to be stored=true for a suggestion > field? Should it be both indexed and stored? > > Kind Regards, > Furkan KAMACI > > On Thu, Dec 22, 2016 at 11:31 AM, Emir Arnautovic < > emir.arnauto...@sematext.com> wrote: > >> That is because my_field_2 is not indexed. >> >> Regards, >> Emir >> >> >> On 21.12.2016 18:04, Furkan KAMACI wrote: >> >>> Hi All, >>> >>> I've a field like that: >>> >>> >> multiValued="false" /> >>> >>> >> stored="true" multiValued="false"/> >>> >>> When I run a suggester on my_field_1 it returns response. However >>> my_field_2 doesn't. I've defined suggester as: >>> >>>suggester >>>FuzzyLookupFactory >>>DocumentDictionaryFactory >>> >>> What can be the reason? >>> >>> Kind Regards, >>> Furkan KAMACI >>> >>> >> -- >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> Solr & Elasticsearch Support * http://sematext.com/ >> >>
Re: Solr Suggester
Hi Emir, As far as I know, it should be enough to be stored=true for a suggestion field? Should it be both indexed and stored? Kind Regards, Furkan KAMACI On Thu, Dec 22, 2016 at 11:31 AM, Emir Arnautovic < emir.arnauto...@sematext.com> wrote: > That is because my_field_2 is not indexed. > > Regards, > Emir > > > On 21.12.2016 18:04, Furkan KAMACI wrote: > >> Hi All, >> >> I've a field like that: >> >> > multiValued="false" /> >> >> > stored="true" multiValued="false"/> >> >> When I run a suggester on my_field_1 it returns response. However >> my_field_2 doesn't. I've defined suggester as: >> >>suggester >>FuzzyLookupFactory >>DocumentDictionaryFactory >> >> What can be the reason? >> >> Kind Regards, >> Furkan KAMACI >> >> > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > >
Re: Solr Suggester
That is because my_field_2 is not indexed. Regards, Emir On 21.12.2016 18:04, Furkan KAMACI wrote: Hi All, I've a field like that: When I run a suggester on my_field_1 it returns response. However my_field_2 doesn't. I've defined suggester as: suggester FuzzyLookupFactory DocumentDictionaryFactory What can be the reason? Kind Regards, Furkan KAMACI -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/
Solr Suggester
Hi All, I've a field like that: When I run a suggester on my_field_1 it returns response. However my_field_2 doesn't. I've defined suggester as: suggester FuzzyLookupFactory DocumentDictionaryFactory What can be the reason? Kind Regards, Furkan KAMACI
Re: Solr Suggester (AnalyzingInfix n BlendedInfix)
Mind you I have no proof that this will cure the problem in 5.2.1, but it seems like it'd be quick to test Good Luck! Erick On Wed, Sep 28, 2016 at 8:51 PM, Manohar Sripada wrote: > Sure Erick! I will try applying the patch. > > Thanks > > On Wednesday, September 28, 2016, Erick Erickson > wrote: > >> AnalyzingInfixSuggester is a mini Solr index, it's working >> as designed by returning the choices you see. I don't think >> you can persuade it to do what you want OOB. >> >> I took a quick look at SOLR-7865 and it's a very simple fix, just >> 3 lines of code change and the rest is test code. Could you >> consider applying that patch to the 5.2.1 code base? and using that >> rather than fully upgrading? >> >> Best, >> Erick >> >> On Wed, Sep 28, 2016 at 4:21 AM, Manohar Sripada > > wrote: >> > I am implementing auto suggestion on Business Name. I >> > used BlendedInfixLookupFactory which worked in all my uses until I >> > encountered into this bug (https://issues.apache.org/ >> jira/browse/SOLR-7865), >> > where suggest.count doesn't work in Solr 5.2.1. But, I can't upgrade >> > anytime soon. :( >> > >> > I tried using AnalyzingInfixSuggester, but, I encountered couple of >> issues >> > with this. Can someone help me with these? >> > >> >1. This lookupImpl is returning duplicate business names ( >> >https://issues.apache.org/jira/browse/LUCENE-6336) in results (the >> data >> >has duplicate business names) which isn't happening >> >with BlendedInfixLookupFactory. I don't want duplicate values. >> >2. Second one is, AnalyzingInfixSuggester is searching on all input >> >keywords - For example, if am looking for "Apple Corporation", it is >> >returning, "Apple Inc", "Apple Corporation", "Oracle Corporation", >> >"Microsoft Corporation". I need the data with only "Apple >> Corporation". >> >Again, this is working fine in BlendedInfixLookupFactory. >> > >> > I don't want fuzzy searches, so, I am not using it. >> > >> > Below are the respective configurations. The fields type uses Standard >> > Tokenizer. >> > >> > >> > businessName_BIF >> > BlendedInfixLookupFactory >> > DocumentDictionaryFactory >> > business_name >> > true >> > true >> > true >> > >> > text_standard >> > true >> > true >> > true >> > suggest_test_business_name_bif >> > 0 >> > false >> > linear >> > 10 >> > >> > revenues >> > id >> > >> > >> > >> > businessName_AIF >> > AnalyzingInfixLookupFactory >> > DocumentDictionaryFactory >> > business_name >> > true >> > true >> > true >> > >> > text_standard >> > true >> > true >> > true >> > suggest_test_business_name_aif >> > 0 >> > true >> > false >> > >> > revenues >> > id >> > >>
Re: Solr Suggester (AnalyzingInfix n BlendedInfix)
Sure Erick! I will try applying the patch. Thanks On Wednesday, September 28, 2016, Erick Erickson wrote: > AnalyzingInfixSuggester is a mini Solr index, it's working > as designed by returning the choices you see. I don't think > you can persuade it to do what you want OOB. > > I took a quick look at SOLR-7865 and it's a very simple fix, just > 3 lines of code change and the rest is test code. Could you > consider applying that patch to the 5.2.1 code base? and using that > rather than fully upgrading? > > Best, > Erick > > On Wed, Sep 28, 2016 at 4:21 AM, Manohar Sripada > wrote: > > I am implementing auto suggestion on Business Name. I > > used BlendedInfixLookupFactory which worked in all my uses until I > > encountered into this bug (https://issues.apache.org/ > jira/browse/SOLR-7865), > > where suggest.count doesn't work in Solr 5.2.1. But, I can't upgrade > > anytime soon. :( > > > > I tried using AnalyzingInfixSuggester, but, I encountered couple of > issues > > with this. Can someone help me with these? > > > >1. This lookupImpl is returning duplicate business names ( > >https://issues.apache.org/jira/browse/LUCENE-6336) in results (the > data > >has duplicate business names) which isn't happening > >with BlendedInfixLookupFactory. I don't want duplicate values. > >2. Second one is, AnalyzingInfixSuggester is searching on all input > >keywords - For example, if am looking for "Apple Corporation", it is > >returning, "Apple Inc", "Apple Corporation", "Oracle Corporation", > >"Microsoft Corporation". I need the data with only "Apple > Corporation". > >Again, this is working fine in BlendedInfixLookupFactory. > > > > I don't want fuzzy searches, so, I am not using it. > > > > Below are the respective configurations. The fields type uses Standard > > Tokenizer. > > > > > > businessName_BIF > > BlendedInfixLookupFactory > > DocumentDictionaryFactory > > business_name > > true > > true > > true > > > > text_standard > > true > > true > > true > > suggest_test_business_name_bif > > 0 > > false > > linear > > 10 > > > > revenues > > id > > > > > > > > businessName_AIF > > AnalyzingInfixLookupFactory > > DocumentDictionaryFactory > > business_name > > true > > true > > true > > > > text_standard > > true > > true > > true > > suggest_test_business_name_aif > > 0 > > true > > false > > > > revenues > > id > > >
Re: Solr Suggester (AnalyzingInfix n BlendedInfix)
AnalyzingInfixSuggester is a mini Solr index, it's working as designed by returning the choices you see. I don't think you can persuade it to do what you want OOB. I took a quick look at SOLR-7865 and it's a very simple fix, just 3 lines of code change and the rest is test code. Could you consider applying that patch to the 5.2.1 code base? and using that rather than fully upgrading? Best, Erick On Wed, Sep 28, 2016 at 4:21 AM, Manohar Sripada wrote: > I am implementing auto suggestion on Business Name. I > used BlendedInfixLookupFactory which worked in all my uses until I > encountered into this bug (https://issues.apache.org/jira/browse/SOLR-7865), > where suggest.count doesn't work in Solr 5.2.1. But, I can't upgrade > anytime soon. :( > > I tried using AnalyzingInfixSuggester, but, I encountered couple of issues > with this. Can someone help me with these? > >1. This lookupImpl is returning duplicate business names ( >https://issues.apache.org/jira/browse/LUCENE-6336) in results (the data >has duplicate business names) which isn't happening >with BlendedInfixLookupFactory. I don't want duplicate values. >2. Second one is, AnalyzingInfixSuggester is searching on all input >keywords - For example, if am looking for "Apple Corporation", it is >returning, "Apple Inc", "Apple Corporation", "Oracle Corporation", >"Microsoft Corporation". I need the data with only "Apple Corporation". >Again, this is working fine in BlendedInfixLookupFactory. > > I don't want fuzzy searches, so, I am not using it. > > Below are the respective configurations. The fields type uses Standard > Tokenizer. > > > businessName_BIF > BlendedInfixLookupFactory > DocumentDictionaryFactory > business_name > true > true > true > > text_standard > true > true > true > suggest_test_business_name_bif > 0 > false > linear > 10 > > revenues > id > > > > businessName_AIF > AnalyzingInfixLookupFactory > DocumentDictionaryFactory > business_name > true > true > true > > text_standard > true > true > true > suggest_test_business_name_aif > 0 > true > false > > revenues > id >
Solr Suggester (AnalyzingInfix n BlendedInfix)
I am implementing auto suggestion on Business Name. I used BlendedInfixLookupFactory which worked in all my uses until I encountered into this bug (https://issues.apache.org/jira/browse/SOLR-7865), where suggest.count doesn't work in Solr 5.2.1. But, I can't upgrade anytime soon. :( I tried using AnalyzingInfixSuggester, but, I encountered couple of issues with this. Can someone help me with these? 1. This lookupImpl is returning duplicate business names ( https://issues.apache.org/jira/browse/LUCENE-6336) in results (the data has duplicate business names) which isn't happening with BlendedInfixLookupFactory. I don't want duplicate values. 2. Second one is, AnalyzingInfixSuggester is searching on all input keywords - For example, if am looking for "Apple Corporation", it is returning, "Apple Inc", "Apple Corporation", "Oracle Corporation", "Microsoft Corporation". I need the data with only "Apple Corporation". Again, this is working fine in BlendedInfixLookupFactory. I don't want fuzzy searches, so, I am not using it. Below are the respective configurations. The fields type uses Standard Tokenizer. businessName_BIF BlendedInfixLookupFactory DocumentDictionaryFactory business_name true true true text_standard true true true suggest_test_business_name_bif 0 false linear 10 revenues id businessName_AIF AnalyzingInfixLookupFactory DocumentDictionaryFactory business_name true true true text_standard true true true suggest_test_business_name_aif 0 true false revenues id
Re: Solr Suggester no results
Bradley:
You're a bloody genius!
That's exactly what I needed to make it work.
For the sake of the archives, after modifying the solrconfig.xml as
indicated and rebuilding the suggester dictionary, the queries started
to kick back results like crazy.
For what it's worth, I'm running Solr 6.1.0.
Thanks so much.
On 8/25/2016 6:48 PM, Bradley Belyeu wrote:
Scott, I’m fairly new to suggesters having just recently built my first one.
But where my configuration differs from yours is on this line:
string
I used the field type name that I had defined instead like:
textSuggest
I’m not certain that would help, but I can’t see where your config is much
different form mine elsewhere.
What version of Solr are you running?
On 8/25/16, 6:41 PM, "Scott Vanderbilt" wrote:
I'm having the exact same problem the O.P. describes from his email back
in May, but my configuration dose not have the same defect his had. So I
am at a loss to understand why my suggest queries are returning no results.
Here is my config:
Relevant bits from schema.xml:
--
...
...
...
Relevant bits from solrconfig.xml:
--
mySuggester
true
10
suggest
...
mySuggester
AnalyzingInfixLookupFactory
suggester_infix_dir
DocumentDictionaryFactory
text_suggest
string
false
false
I build the dictionary like this:
http://example.com:8983/solr/rrib/suggest?suggest.build=true
and get back this response:
0
964
build
I then attempt a query like this:
http://example.com:8983/solr/rrib/suggest?q=re
and get back this response:
0
37
0
In Solr's Admin interface, I see the following under OTHER->suggest on
Plugins/Stats page:
class:suggest
description:Suggester component
src:
version:6.1.0
stats:
mySuggester: SolrSuggester [ name=mySuggester,
sourceLocation=null, storeDir=,
lookupImpl=AnalyzingInfixLookupFactory,
dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ]
totalSizeInBytes: 6795
The value of 6,795 bytes seems pretty small to me for a repository of
403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue
that the dictionary has not been fully populated, which probably
explains the empty result sets, but I cannot figure out why
Any assistance would be gratefully received.
Thank you.
On 5/6/2016 9:42 AM, Erick Erickson wrote:
> First off, kudos for providing the details, that really helps!
>
> The root of your problem is that your suggest field has stored="false".
> DocumentDictionaryFactory reads through all the
> docs in your corpus, extracts the stored data and puts it in the FST.
Since
> you don't have any stored data your FST is...er...minimal.
>
> I'd also add
> suggester_fuzzy_dir
> to the searchComponent. You'll find the FST on disk in that directory
where it
> can be read next time Solr starts up. It is also helpful for figuring out
> whether there are suggestions to be had.
>
> And a minor nit, you probably don't want to specify suggest.dictionary
> in your query,
> that's already specified in your config.
>
> And it looks like you're alive to the fact that with that setup
> capitalization matters
> as does the fact that these suggestions be matched from the beginning of
the
> field...
>
> Best,
> Erick
>
> On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
> wrote:
>> Hi there,
>>
>> I want to use the Solr suggester component for city names. I have the
>> following settings:
>> schema.xml
>>
>> Field definition
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> The field i want to apply the suggester on
>>
>>
>>
>> The copy field
>>
>>
>>
>> The field
>>
>>
>>
>> solr-config.xml
>>
>>
Re: Solr Suggester no results
Scott, I’m fairly new to suggesters having just recently built my first one. But where my configuration differs from yours is on this line: string I used the field type name that I had defined instead like: textSuggest I’m not certain that would help, but I can’t see where your config is much different form mine elsewhere. What version of Solr are you running? On 8/25/16, 6:41 PM, "Scott Vanderbilt" wrote: I'm having the exact same problem the O.P. describes from his email back in May, but my configuration dose not have the same defect his had. So I am at a loss to understand why my suggest queries are returning no results. Here is my config: Relevant bits from schema.xml: -- ... ... ... Relevant bits from solrconfig.xml: -- mySuggester true 10 suggest ... mySuggester AnalyzingInfixLookupFactory suggester_infix_dir DocumentDictionaryFactory text_suggest string false false I build the dictionary like this: http://example.com:8983/solr/rrib/suggest?suggest.build=true and get back this response: 0 964 build I then attempt a query like this: http://example.com:8983/solr/rrib/suggest?q=re and get back this response: 0 37 0 In Solr's Admin interface, I see the following under OTHER->suggest on Plugins/Stats page: class:suggest description:Suggester component src: version:6.1.0 stats: mySuggester: SolrSuggester [ name=mySuggester, sourceLocation=null, storeDir=, lookupImpl=AnalyzingInfixLookupFactory, dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ] totalSizeInBytes: 6795 The value of 6,795 bytes seems pretty small to me for a repository of 403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue that the dictionary has not been fully populated, which probably explains the empty result sets, but I cannot figure out why Any assistance would be gratefully received. Thank you. On 5/6/2016 9:42 AM, Erick Erickson wrote: > First off, kudos for providing the details, that really helps! > > The root of your problem is that your suggest field has stored="false". > DocumentDictionaryFactory reads through all the > docs in your corpus, extracts the stored data and puts it in the FST. Since > you don't have any stored data your FST is...er...minimal. > > I'd also add > suggester_fuzzy_dir > to the searchComponent. You'll find the FST on disk in that directory where it > can be read next time Solr starts up. It is also helpful for figuring out > whether there are suggestions to be had. > > And a minor nit, you probably don't want to specify suggest.dictionary > in your query, > that's already specified in your config. > > And it looks like you're alive to the fact that with that setup > capitalization matters > as does the fact that these suggestions be matched from the beginning of the > field... > > Best, > Erick > > On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos > wrote: >> Hi there, >> >> I want to use the Solr suggester component for city names. I have the >> following settings: >> schema.xml >> >> Field definition >> >> >> >> >> >> >> >> >> >> The field i want to apply the suggester on >> >> >> >> The copy field >> >> >> >> The field >> >> >> >> solr-config.xml >> >> >> >> true >> 10 >> mySuggester >> >> >> suggest >> >> >> >> >> >> >> >> mySuggester >> FuzzyLookupFactory
Re: Solr Suggester no results
I'm having the exact same problem the O.P. describes from his email back
in May, but my configuration dose not have the same defect his had. So I
am at a loss to understand why my suggest queries are returning no results.
Here is my config:
Relevant bits from schema.xml:
--
multiValued="true"/>
stored="true" multiValued="true"/>
...
stored="true" multiValued="true" />
...
...
positionIncrementGap="100">
Relevant bits from solrconfig.xml:
--
mySuggester
true
10
suggest
...
mySuggester
AnalyzingInfixLookupFactory
suggester_infix_dir
DocumentDictionaryFactory
text_suggest
string
false
false
I build the dictionary like this:
http://example.com:8983/solr/rrib/suggest?suggest.build=true
and get back this response:
0
964
build
I then attempt a query like this:
http://example.com:8983/solr/rrib/suggest?q=re
and get back this response:
0
37
0
In Solr's Admin interface, I see the following under OTHER->suggest on
Plugins/Stats page:
class:suggest
description:Suggester component
src:
version:6.1.0
stats:
mySuggester: SolrSuggester [ name=mySuggester,
sourceLocation=null, storeDir=,
lookupImpl=AnalyzingInfixLookupFactory,
dictionaryImpl=DocumentDictionaryFactory, sizeInBytes=6795 ]
totalSizeInBytes: 6795
The value of 6,795 bytes seems pretty small to me for a repository of
403 XML files containing about 1.5 MB of mark-up. Perhaps that is a clue
that the dictionary has not been fully populated, which probably
explains the empty result sets, but I cannot figure out why
Any assistance would be gratefully received.
Thank you.
On 5/6/2016 9:42 AM, Erick Erickson wrote:
First off, kudos for providing the details, that really helps!
The root of your problem is that your suggest field has stored="false".
DocumentDictionaryFactory reads through all the
docs in your corpus, extracts the stored data and puts it in the FST. Since
you don't have any stored data your FST is...er...minimal.
I'd also add
suggester_fuzzy_dir
to the searchComponent. You'll find the FST on disk in that directory where it
can be read next time Solr starts up. It is also helpful for figuring out
whether there are suggestions to be had.
And a minor nit, you probably don't want to specify suggest.dictionary
in your query,
that's already specified in your config.
And it looks like you're alive to the fact that with that setup
capitalization matters
as does the fact that these suggestions be matched from the beginning of the
field...
Best,
Erick
On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
wrote:
Hi there,
I want to use the Solr suggester component for city names. I have the
following settings:
schema.xml
Field definition
The field i want to apply the suggester on
The copy field
The field
solr-config.xml
true
10
mySuggester
suggest
mySuggester
FuzzyLookupFactory
DocumentDictionaryFactory
citySuggest
string
Then i run
http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
to build the suggest component
Finally i run
http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
but i get an empty result set
{"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]
Are there any obvious mistakes? Any thoughts?
Is there a way to filter the results based on weight - SOLR suggester?
Hi, I am using suggester component in SOLR 5.5.1 and sort the matching suggestion based on a custom field (lookupCount) field. The below configuration seems to work fine but its returning the matching term even if the weight is set to 0. Is there a way to restrict returning the matching term based on weight field? Something like. Return the matching suggestions that doesn't have the value of the weight field set to 0? mySuggester FuzzyLookupFactory DocumentDictionaryFactory userQuery lookupCount string *Current sample output (returns the matching term even it has weight set to 0) * /solr/test/suggest?suggest=true&suggest.dictionary=mySuggester&wt=xml&suggest.q=bill&suggest.count=5 5 bill 0 billing history 69753 -- View this message in context: http://lucene.472066.n3.nabble.com/Is-there-a-way-to-filter-the-results-based-on-weight-SOLR-suggester-tp4289286.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: SOLR Suggester | CFQ Variable | - not supported
Hi, Not by getting desired output i mean is that I am not able to filter out the result on the basis of CFQ parameter, and in case I am querying SOLR with CFQ value to be as normal string, i.e. no special character, it is working fine. Do let me know in case more info required. For me my main problem is how can I pass special charater like "-" in CFQ so that filtering will work. Thanks, On Wed, Jul 20, 2016 at 7:16 PM, Alessandro Benedetti wrote: > Rajesh, > I would definitely suggest you to debug your use case or at least to give > much more details : > > "is not giving me desired output" is basically useless, try to describe > what was your desidered output, what you get instead, if you have any log, > ect... > > Cheers > > On Tue, Jul 19, 2016 at 8:42 AM, Rajesh Kapur > wrote: > > > Hi, > > > > Any update on this please. > > > > Thanks. > > > > On Sun, Jul 17, 2016 at 9:25 PM, Rajesh Kapur > > wrote: > > > > > Hi, > > > > > > Thanks for the reply. > > > > > > Yes, i tried to search by setting CFQ=abc\-def and also as "abc-def", > but > > > no luck. > > > > > > Thanks. > > > > > > On Sun, Jul 17, 2016 at 9:19 PM, Erick Erickson < > erickerick...@gmail.com > > > > > > wrote: > > > > > >> What have you tried? Did you try escaping it? > > >> > > >> Best, > > >> Erick > > >> > > >> On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur < > rajeshkapu...@gmail.com> > > >> wrote: > > >> > Hi > > >> > > > >> > I am still facing this issue. Anyone can help me in this. > > >> > > > >> > Thanks > > >> > > > >> > On 14-Jul-2016 1:15 PM, "Rajesh Kapur" > > wrote: > > >> > > > >> >> Hi, > > >> >> > > >> >> I am facing issue while implementing suggester for my project > where I > > >> am > > >> >> passing CFQ value having - in between, but it is not giving me > > desired > > >> >> output. > > >> >> > > >> >> Could you please let me know what should I do so that SOLR > suggester > > >> >> starts accepting - in CFQ parameters. > > >> >> > > >> >> Thanks > > >> >> Rajesh Kapur > > >> >> > > >> > > > > > > > > > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England >
Re: SOLR Suggester | CFQ Variable | - not supported
Rajesh, I would definitely suggest you to debug your use case or at least to give much more details : "is not giving me desired output" is basically useless, try to describe what was your desidered output, what you get instead, if you have any log, ect... Cheers On Tue, Jul 19, 2016 at 8:42 AM, Rajesh Kapur wrote: > Hi, > > Any update on this please. > > Thanks. > > On Sun, Jul 17, 2016 at 9:25 PM, Rajesh Kapur > wrote: > > > Hi, > > > > Thanks for the reply. > > > > Yes, i tried to search by setting CFQ=abc\-def and also as "abc-def", but > > no luck. > > > > Thanks. > > > > On Sun, Jul 17, 2016 at 9:19 PM, Erick Erickson > > > wrote: > > > >> What have you tried? Did you try escaping it? > >> > >> Best, > >> Erick > >> > >> On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur > >> wrote: > >> > Hi > >> > > >> > I am still facing this issue. Anyone can help me in this. > >> > > >> > Thanks > >> > > >> > On 14-Jul-2016 1:15 PM, "Rajesh Kapur" > wrote: > >> > > >> >> Hi, > >> >> > >> >> I am facing issue while implementing suggester for my project where I > >> am > >> >> passing CFQ value having - in between, but it is not giving me > desired > >> >> output. > >> >> > >> >> Could you please let me know what should I do so that SOLR suggester > >> >> starts accepting - in CFQ parameters. > >> >> > >> >> Thanks > >> >> Rajesh Kapur > >> >> > >> > > > > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: SOLR Suggester | CFQ Variable | - not supported
Hi, Any update on this please. Thanks. On Sun, Jul 17, 2016 at 9:25 PM, Rajesh Kapur wrote: > Hi, > > Thanks for the reply. > > Yes, i tried to search by setting CFQ=abc\-def and also as "abc-def", but > no luck. > > Thanks. > > On Sun, Jul 17, 2016 at 9:19 PM, Erick Erickson > wrote: > >> What have you tried? Did you try escaping it? >> >> Best, >> Erick >> >> On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur >> wrote: >> > Hi >> > >> > I am still facing this issue. Anyone can help me in this. >> > >> > Thanks >> > >> > On 14-Jul-2016 1:15 PM, "Rajesh Kapur" wrote: >> > >> >> Hi, >> >> >> >> I am facing issue while implementing suggester for my project where I >> am >> >> passing CFQ value having - in between, but it is not giving me desired >> >> output. >> >> >> >> Could you please let me know what should I do so that SOLR suggester >> >> starts accepting - in CFQ parameters. >> >> >> >> Thanks >> >> Rajesh Kapur >> >> >> > >
Re: SOLR Suggester | CFQ Variable | - not supported
Hi, Thanks for the reply. Yes, i tried to search by setting CFQ=abc\-def and also as "abc-def", but no luck. Thanks. On Sun, Jul 17, 2016 at 9:19 PM, Erick Erickson wrote: > What have you tried? Did you try escaping it? > > Best, > Erick > > On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur > wrote: > > Hi > > > > I am still facing this issue. Anyone can help me in this. > > > > Thanks > > > > On 14-Jul-2016 1:15 PM, "Rajesh Kapur" wrote: > > > >> Hi, > >> > >> I am facing issue while implementing suggester for my project where I am > >> passing CFQ value having - in between, but it is not giving me desired > >> output. > >> > >> Could you please let me know what should I do so that SOLR suggester > >> starts accepting - in CFQ parameters. > >> > >> Thanks > >> Rajesh Kapur > >> >
Re: SOLR Suggester | CFQ Variable | - not supported
What have you tried? Did you try escaping it? Best, Erick On Sun, Jul 17, 2016 at 7:32 AM, Rajesh Kapur wrote: > Hi > > I am still facing this issue. Anyone can help me in this. > > Thanks > > On 14-Jul-2016 1:15 PM, "Rajesh Kapur" wrote: > >> Hi, >> >> I am facing issue while implementing suggester for my project where I am >> passing CFQ value having - in between, but it is not giving me desired >> output. >> >> Could you please let me know what should I do so that SOLR suggester >> starts accepting - in CFQ parameters. >> >> Thanks >> Rajesh Kapur >>
Re: SOLR Suggester | CFQ Variable | - not supported
Hi I am still facing this issue. Anyone can help me in this. Thanks On 14-Jul-2016 1:15 PM, "Rajesh Kapur" wrote: > Hi, > > I am facing issue while implementing suggester for my project where I am > passing CFQ value having - in between, but it is not giving me desired > output. > > Could you please let me know what should I do so that SOLR suggester > starts accepting - in CFQ parameters. > > Thanks > Rajesh Kapur >
SOLR Suggester | CFQ Variable | - not supported
Hi, I am facing issue while implementing suggester for my project where I am passing CFQ value having - in between, but it is not giving me desired output. Could you please let me know what should I do so that SOLR suggester starts accepting - in CFQ parameters. Thanks Rajesh Kapur
Re: Multiple context field / filters in Solr suggester
Anyone ? -- View this message in context: http://lucene.472066.n3.nabble.com/Multiple-context-field-filters-in-Solr-suggester-tp4283739p4283894.html Sent from the Solr - User mailing list archive at Nabble.com.
Multiple context field / filters in Solr suggester
Hi, Just trying to understand if Solr suggester supports multiple filtering through the "contextField" option. As shown in the config below, is it possible to have two contextFields defined where I can use "cat" and "manu" as filtering criteria on the suggested result ? mySuggester AnalyzingInfixLookupFactory DocumentDictionaryFactory name price cat manu string false The only reference around this seemed to be Solr-7888, but based on my understanding, it talks about boolean support for a given context. Any pointers will be appreciated. Thanks, Shamik
Re: Solr Suggester no results
Yes, i also realized that stored="false" was the problem..It is also
stated clearly in the documentation : "To be used as the basis for a
suggestion, the field must be stored."
Thanks for your time,
Grigoris
2016-05-06 19:42 GMT+03:00 Erick Erickson :
> First off, kudos for providing the details, that really helps!
>
> The root of your problem is that your suggest field has stored="false".
> DocumentDictionaryFactory reads through all the
> docs in your corpus, extracts the stored data and puts it in the FST. Since
> you don't have any stored data your FST is...er...minimal.
>
> I'd also add
> suggester_fuzzy_dir
> to the searchComponent. You'll find the FST on disk in that directory
> where it
> can be read next time Solr starts up. It is also helpful for figuring out
> whether there are suggestions to be had.
>
> And a minor nit, you probably don't want to specify suggest.dictionary
> in your query,
> that's already specified in your config.
>
> And it looks like you're alive to the fact that with that setup
> capitalization matters
> as does the fact that these suggestions be matched from the beginning of
> the
> field...
>
> Best,
> Erick
>
> On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
> wrote:
> > Hi there,
> >
> > I want to use the Solr suggester component for city names. I have the
> > following settings:
> > schema.xml
> >
> > Field definition
> >
> > positionIncrementGap="100">
> >
> >
> >
> >
> >
> >
> >
> > The field i want to apply the suggester on
> >
> >
> >
> > The copy field
> >
> >
> >
> > The field
> >
> > indexed="true" />
> >
> > solr-config.xml
> >
> > startup="lazy">
> >
> > true
> > 10
> > mySuggester
> >
> >
> > suggest
> >
> >
> >
> >
> >
> >
> >
> > mySuggester
> > FuzzyLookupFactory
> > DocumentDictionaryFactory
> > citySuggest
> > string
> >
> >
> >
> > Then i run
> >
> >
> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
> >
> > to build the suggest component
> >
> > Finally i run
> >
> >
> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
> >
> > but i get an empty result set
> >
> >
> {"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]
> >
> > Are there any obvious mistakes? Any thoughts?
>
Re: Solr Suggester no results
First off, kudos for providing the details, that really helps!
The root of your problem is that your suggest field has stored="false".
DocumentDictionaryFactory reads through all the
docs in your corpus, extracts the stored data and puts it in the FST. Since
you don't have any stored data your FST is...er...minimal.
I'd also add
suggester_fuzzy_dir
to the searchComponent. You'll find the FST on disk in that directory where it
can be read next time Solr starts up. It is also helpful for figuring out
whether there are suggestions to be had.
And a minor nit, you probably don't want to specify suggest.dictionary
in your query,
that's already specified in your config.
And it looks like you're alive to the fact that with that setup
capitalization matters
as does the fact that these suggestions be matched from the beginning of the
field...
Best,
Erick
On Thu, May 5, 2016 at 1:05 AM, Grigoris Iliopoulos
wrote:
> Hi there,
>
> I want to use the Solr suggester component for city names. I have the
> following settings:
> schema.xml
>
> Field definition
>
> positionIncrementGap="100">
>
>
>
>
>
>
>
> The field i want to apply the suggester on
>
>
>
> The copy field
>
>
>
> The field
>
>
>
> solr-config.xml
>
>
>
> true
> 10
> mySuggester
>
>
> suggest
>
>
>
>
>
>
>
> mySuggester
> FuzzyLookupFactory
> DocumentDictionaryFactory
> citySuggest
> string
>
>
>
> Then i run
>
> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
>
> to build the suggest component
>
> Finally i run
>
>
> http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
>
> but i get an empty result set
>
> {"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]
>
> Are there any obvious mistakes? Any thoughts?
Solr Suggester no results
Hi there,
I want to use the Solr suggester component for city names. I have the
following settings:
schema.xml
Field definition
The field i want to apply the suggester on
The copy field
The field
solr-config.xml
true
10
mySuggester
suggest
mySuggester
FuzzyLookupFactory
DocumentDictionaryFactory
citySuggest
string
Then i run
http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath&suggest.build=true
to build the suggest component
Finally i run
http://localhost:8983/solr/company/suggest?suggest=true&suggest.dictionary=mySuggester&wt=json&suggest.q=Ath
but i get an empty result set
{"responseHeader":{"status":0,"QTime":0},"suggest":{"mySuggester":{"Ath":{"numFound":0,"suggestions":[]
Are there any obvious mistakes? Any thoughts?
Re: [Solr Suggester Component] Unique suggestions
Hi Roland, I agree with you, I updated the Jira issue and I will spend some more time on that. I think the best solution should be Solr core side, I will sort out some proposal, that we can discuss ( probably I will use the related spin off Jira issue, linked in : https://issues.apache.org/jira/browse/LUCENE-6336 On 12 March 2016 at 08:10, Roland Szűcs wrote: > Hi all, > > I think nobody can question that this configuration parameter is > neccessary. Most of the use cases need distinct values. > > I use solr 5.2.1 for an ebook store. I use the suggester component for > autocomplete titles, authors, publishers. Some of our publishers have > thousand of ebooks. > > In theese cases I got back a json with thousands of identical cases which > obviously not what I want. > > Cheers, > Roland Szucs > > > 2016. márc. 11. dátummal, 17:46 időpontban Alessandro Benedetti < > abenede...@apache.org> írta: > > > No one didn't care of the topic, let me try in the solr-user list as > well ! > > Does anyone think that should exist a parameter that allow a Suggester to > > not return duplicate suggestions ? > > In my opinion could be a good improvement ! > > In the initial patch i was acting at SolrJ level, but to be honest I > think > > the enhancement should be done internally in Solr. > > let me know your opinion and I can proceed with a patch. > > > > Cheers > > > > On 16 February 2016 at 16:19, Alessandro Benedetti < > abenede...@apache.org> > > wrote: > > > >> It has been some time I was wondering why in a lot of scenarios we > return > >> duplicates per suggestions. > >> Apart the rare scenarios where someone is associating a payload to each > >> suggestion, suggestions with the same label are practically > >> undistinguishable from a Human perspective. > >> I would suggest to add a configuration parameter for the component to > >> avoid duplicates when the parameter is true. > >> > >> I submitted a patch for SolrJ but now I think the problem should be > solved > >> in the Solr core and the possibility of configuring the strategy given > to > >> the user at Solrconfig level. > >> If you agree I will create a new specific Jira issue to solve the > problem > >> in Solr itself. > >> > >> SolrJ patch > >> https://issues.apache.org/jira/browse/SOLR-8672 > >> > >> Cheers > >> > >> -- > >> -- > >> > >> Benedetti Alessandro > >> Visiting card : http://about.me/alessandro_benedetti > >> > >> "Tyger, tyger burning bright > >> In the forests of the night, > >> What immortal hand or eye > >> Could frame thy fearful symmetry?" > >> > >> William Blake - Songs of Experience -1794 England > > > > > > > > -- > > -- > > > > Benedetti Alessandro > > Visiting card : http://about.me/alessandro_benedetti > > > > "Tyger, tyger burning bright > > In the forests of the night, > > What immortal hand or eye > > Could frame thy fearful symmetry?" > > > > William Blake - Songs of Experience -1794 England > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: [Solr Suggester Component] Unique suggestions
Hi all, I think nobody can question that this configuration parameter is neccessary. Most of the use cases need distinct values. I use solr 5.2.1 for an ebook store. I use the suggester component for autocomplete titles, authors, publishers. Some of our publishers have thousand of ebooks. In theese cases I got back a json with thousands of identical cases which obviously not what I want. Cheers, Roland Szucs 2016. márc. 11. dátummal, 17:46 időpontban Alessandro Benedetti írta: > No one didn't care of the topic, let me try in the solr-user list as well ! > Does anyone think that should exist a parameter that allow a Suggester to > not return duplicate suggestions ? > In my opinion could be a good improvement ! > In the initial patch i was acting at SolrJ level, but to be honest I think > the enhancement should be done internally in Solr. > let me know your opinion and I can proceed with a patch. > > Cheers > > On 16 February 2016 at 16:19, Alessandro Benedetti > wrote: > >> It has been some time I was wondering why in a lot of scenarios we return >> duplicates per suggestions. >> Apart the rare scenarios where someone is associating a payload to each >> suggestion, suggestions with the same label are practically >> undistinguishable from a Human perspective. >> I would suggest to add a configuration parameter for the component to >> avoid duplicates when the parameter is true. >> >> I submitted a patch for SolrJ but now I think the problem should be solved >> in the Solr core and the possibility of configuring the strategy given to >> the user at Solrconfig level. >> If you agree I will create a new specific Jira issue to solve the problem >> in Solr itself. >> >> SolrJ patch >> https://issues.apache.org/jira/browse/SOLR-8672 >> >> Cheers >> >> -- >> -- >> >> Benedetti Alessandro >> Visiting card : http://about.me/alessandro_benedetti >> >> "Tyger, tyger burning bright >> In the forests of the night, >> What immortal hand or eye >> Could frame thy fearful symmetry?" >> >> William Blake - Songs of Experience -1794 England > > > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England
Re: [Solr Suggester Component] Unique suggestions
No one didn't care of the topic, let me try in the solr-user list as well ! Does anyone think that should exist a parameter that allow a Suggester to not return duplicate suggestions ? In my opinion could be a good improvement ! In the initial patch i was acting at SolrJ level, but to be honest I think the enhancement should be done internally in Solr. let me know your opinion and I can proceed with a patch. Cheers On 16 February 2016 at 16:19, Alessandro Benedetti wrote: > It has been some time I was wondering why in a lot of scenarios we return > duplicates per suggestions. > Apart the rare scenarios where someone is associating a payload to each > suggestion, suggestions with the same label are practically > undistinguishable from a Human perspective. > I would suggest to add a configuration parameter for the component to > avoid duplicates when the parameter is true. > > I submitted a patch for SolrJ but now I think the problem should be solved > in the Solr core and the possibility of configuring the strategy given to > the user at Solrconfig level. > If you agree I will create a new specific Jira issue to solve the problem > in Solr itself. > > SolrJ patch > https://issues.apache.org/jira/browse/SOLR-8672 > > Cheers > > -- > -- > > Benedetti Alessandro > Visiting card : http://about.me/alessandro_benedetti > > "Tyger, tyger burning bright > In the forests of the night, > What immortal hand or eye > Could frame thy fearful symmetry?" > > William Blake - Songs of Experience -1794 England > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Suggester with Geo?
Out of the box the Suggester componenent provides a set of Lookup implementation to look for the suggestions from the dictionary selected. Most of the suggestion lookups are FST based ( except the Analysing Infix one , that relies on an external auxiliary Lucene Index) . Depending on your requirement you should customise the lookUp implementation to score the suggestions taking in consideration the distance. Customising the "Auxiliary Lucene Index " based look up strategies would be easier. Playing with the FST look up would be much more complicated. Cheers On 9 November 2015 at 20:16, Sameer Maggon wrote: > Looking through the code and some example Suggesters, it seems that > theoretically, one can write a GeoSuggester and provide that as the Lookup > implementation (lookupimpl) that would factor in the geo score or extend > the SolrSuggestor to support spatial extensions in the same spirit as > "Filters" are supported today. > > Sameer. > > On Mon, Nov 9, 2015 at 11:47 AM, William Bell wrote: > > > Yeah we have that working today. But the issue is we want to use > > http://lucidworks.com/blog/solr-suggester/ > > > > And you cannot do a boost with that right? > > > > > > > > On Mon, Nov 9, 2015 at 12:41 PM, Sameer Maggon < > sam...@measuredsearch.com> > > wrote: > > > > > Have you looked at the Spatial extensions for Solr? If you are indexing > > > Lat/Lon along with your documents, you can compute the distance from > the > > > origin & use that distance as one of the boost factors to affect the > > score. > > > Typically, use cases around that combine the geo score with other > factors > > > as a pure sort by geo score might not give you the relevant results. > > > > > > e.g. typing to search for "sushi restaurants" near Santa Monica, CA - > you > > > might not want "thai restaurants" that are closest to you. (Local > Search > > > use case) > > > > > > https://cwiki.apache.org/confluence/display/solr/Spatial+Search > > > > > > Thanks, > > > -- > > > *Sameer Maggon* > > > www.measuredsearch.com <http://measuredsearch.com/> > > > Fully Managed Solr-as-a-Service | Solr Consulting | Solr Support > > > > > > > > > > > > On Mon, Nov 9, 2015 at 11:18 AM, William Bell > > wrote: > > > > > > > http://lucidworks.com/blog/solr-suggester/ > > > > > > > > > > > > Wondering if anyone has uses these new techniques with a boost on > > > > geodist() inverted? So the rows that get returned that are closest > > > > need to come back first. > > > > > > > > > > > > We are still using Edge Grams since we have not figured out how to > > > > boost the results on geo spatial. > > > > > > > > > > > > Anyone have thoughts? > > > > > > > > > > > > > > > > > > > > -- > > > > Bill Bell > > > > billnb...@gmail.com > > > > cell 720-256-8076 > > > > > > > > > > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Suggester with Geo?
Hello, It seems it's nearly possible, but needs to be done https://github.com/apache/lucene-solr/blob/trunk/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java#L576 (this is for AnalyzingInfixLookupFactory) And for FuzzySuggester such implementation is much more complicated. On Mon, Nov 9, 2015 at 10:18 PM, William Bell wrote: > http://lucidworks.com/blog/solr-suggester/ > > > Wondering if anyone has uses these new techniques with a boost on > geodist() inverted? So the rows that get returned that are closest > need to come back first. > > > We are still using Edge Grams since we have not figured out how to > boost the results on geo spatial. > > > Anyone have thoughts? > > > > > -- > Bill Bell > billnb...@gmail.com > cell 720-256-8076 > -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics <http://www.griddynamics.com>
Re: Solr Suggester with Geo?
Looking through the code and some example Suggesters, it seems that theoretically, one can write a GeoSuggester and provide that as the Lookup implementation (lookupimpl) that would factor in the geo score or extend the SolrSuggestor to support spatial extensions in the same spirit as "Filters" are supported today. Sameer. On Mon, Nov 9, 2015 at 11:47 AM, William Bell wrote: > Yeah we have that working today. But the issue is we want to use > http://lucidworks.com/blog/solr-suggester/ > > And you cannot do a boost with that right? > > > > On Mon, Nov 9, 2015 at 12:41 PM, Sameer Maggon > wrote: > > > Have you looked at the Spatial extensions for Solr? If you are indexing > > Lat/Lon along with your documents, you can compute the distance from the > > origin & use that distance as one of the boost factors to affect the > score. > > Typically, use cases around that combine the geo score with other factors > > as a pure sort by geo score might not give you the relevant results. > > > > e.g. typing to search for "sushi restaurants" near Santa Monica, CA - you > > might not want "thai restaurants" that are closest to you. (Local Search > > use case) > > > > https://cwiki.apache.org/confluence/display/solr/Spatial+Search > > > > Thanks, > > -- > > *Sameer Maggon* > > www.measuredsearch.com <http://measuredsearch.com/> > > Fully Managed Solr-as-a-Service | Solr Consulting | Solr Support > > > > > > > > On Mon, Nov 9, 2015 at 11:18 AM, William Bell > wrote: > > > > > http://lucidworks.com/blog/solr-suggester/ > > > > > > > > > Wondering if anyone has uses these new techniques with a boost on > > > geodist() inverted? So the rows that get returned that are closest > > > need to come back first. > > > > > > > > > We are still using Edge Grams since we have not figured out how to > > > boost the results on geo spatial. > > > > > > > > > Anyone have thoughts? > > > > > > > > > > > > > > > -- > > > Bill Bell > > > billnb...@gmail.com > > > cell 720-256-8076 > > > > > >
Re: Solr Suggester with Geo?
Yeah we have that working today. But the issue is we want to use http://lucidworks.com/blog/solr-suggester/ And you cannot do a boost with that right? On Mon, Nov 9, 2015 at 12:41 PM, Sameer Maggon wrote: > Have you looked at the Spatial extensions for Solr? If you are indexing > Lat/Lon along with your documents, you can compute the distance from the > origin & use that distance as one of the boost factors to affect the score. > Typically, use cases around that combine the geo score with other factors > as a pure sort by geo score might not give you the relevant results. > > e.g. typing to search for "sushi restaurants" near Santa Monica, CA - you > might not want "thai restaurants" that are closest to you. (Local Search > use case) > > https://cwiki.apache.org/confluence/display/solr/Spatial+Search > > Thanks, > -- > *Sameer Maggon* > www.measuredsearch.com <http://measuredsearch.com/> > Fully Managed Solr-as-a-Service | Solr Consulting | Solr Support > > > > On Mon, Nov 9, 2015 at 11:18 AM, William Bell wrote: > > > http://lucidworks.com/blog/solr-suggester/ > > > > > > Wondering if anyone has uses these new techniques with a boost on > > geodist() inverted? So the rows that get returned that are closest > > need to come back first. > > > > > > We are still using Edge Grams since we have not figured out how to > > boost the results on geo spatial. > > > > > > Anyone have thoughts? > > > > > > > > > > -- > > Bill Bell > > billnb...@gmail.com > > cell 720-256-8076 > > > -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: Solr Suggester with Geo?
Have you looked at the Spatial extensions for Solr? If you are indexing Lat/Lon along with your documents, you can compute the distance from the origin & use that distance as one of the boost factors to affect the score. Typically, use cases around that combine the geo score with other factors as a pure sort by geo score might not give you the relevant results. e.g. typing to search for "sushi restaurants" near Santa Monica, CA - you might not want "thai restaurants" that are closest to you. (Local Search use case) https://cwiki.apache.org/confluence/display/solr/Spatial+Search Thanks, -- *Sameer Maggon* www.measuredsearch.com <http://measuredsearch.com/> Fully Managed Solr-as-a-Service | Solr Consulting | Solr Support On Mon, Nov 9, 2015 at 11:18 AM, William Bell wrote: > http://lucidworks.com/blog/solr-suggester/ > > > Wondering if anyone has uses these new techniques with a boost on > geodist() inverted? So the rows that get returned that are closest > need to come back first. > > > We are still using Edge Grams since we have not figured out how to > boost the results on geo spatial. > > > Anyone have thoughts? > > > > > -- > Bill Bell > billnb...@gmail.com > cell 720-256-8076 >
Solr Suggester with Geo?
http://lucidworks.com/blog/solr-suggester/ Wondering if anyone has uses these new techniques with a boost on geodist() inverted? So the rows that get returned that are closest need to come back first. We are still using Edge Grams since we have not figured out how to boost the results on geo spatial. Anyone have thoughts? -- Bill Bell billnb...@gmail.com cell 720-256-8076
Re: Solr suggester throws error on core reload.
Hi Erick, Sorry for the confusion caused, Next time will be more careful while posting questions in forum. Actually we are using AnalyzingInfixLookupFactory for auto-suggestions. And currently is has open issue with core reload (https://issues.apache.org/jira/browse/SOLR-6246). So my question was related to resolution of this issue. - Nutch Solr User "The ultimate search engine would basically understand everything in the world, and it would always give you the right thing." -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-suggester-throws-error-on-core-reload-tp4220725p4223098.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr suggester throws error on core reload.
You might review: http://wiki.apache.org/solr/UsingMailingLists You've provide exactly no information that would help us help you. Or did you hijack the thread? Because the title has nothing to do with the question. See: http://people.apache.org/~hossman/#threadhijack When starting a new discussion on a mailing list, please do not reply to an existing message, instead start a fresh email. Even if you change the subject line of your email, other mail headers still track which thread you replied to and your question is "hidden" in that thread and gets less attention. It makes following discussions in the mailing list archives particularly difficult. And assuming your question is important here and not the title, see: http://lucidworks.com/blog/solr-suggester/ Best, Erick On Thu, Aug 13, 2015 at 11:33 PM, Nutch Solr User wrote: > I want to use AnalyzingInfixLookupFactory for my autosuggestions. > > Any idea when this issue will get fixed? Do we have any workaround for this > issue. > > > > - > Nutch Solr User > > "The ultimate search engine would basically understand everything in the > world, and it would always give you the right thing." > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-suggester-throws-error-on-core-reload-tp4220725p4222902.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr suggester throws error on core reload.
I want to use AnalyzingInfixLookupFactory for my autosuggestions. Any idea when this issue will get fixed? Do we have any workaround for this issue. - Nutch Solr User "The ultimate search engine would basically understand everything in the world, and it would always give you the right thing." -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-suggester-throws-error-on-core-reload-tp4220725p4222902.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr suggester throws error on core reload.
I found existing issue here https://issues.apache.org/jira/browse/SOLR-6246 . It says fix version 5.2 but Resolution is unresolved. - Nutch Solr User "The ultimate search engine would basically understand everything in the world, and it would always give you the right thing." -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-suggester-throws-error-on-core-reload-tp4220725p4220730.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr suggester throws error on core reload.
I am using AnalyzingInfixSuggester for auto-suggest feature. but whenever I try to reload solr core following error is thrown , org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@E:\SSearch\SolrServer\solr-5.2.1\server\solr\ssearch\data\main-suggest\write.lock After restart everything works fine. What could be the reason for this? - Nutch Solr User "The ultimate search engine would basically understand everything in the world, and it would always give you the right thing." -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-suggester-throws-error-on-core-reload-tp4220725.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester not working.
Vincenzo D'Amore, Yes You are right, it's a typo, I missed it while cleaning the XML to put on the Solr-User list. But, *REMOVE *the following line, this was not used in my Solr 5.1 configuration: * false--> * Regards, Sachin Vyas. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086p4214945.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester not working.
Thanks Sachin Vyas. Maybe I have found a typo, but there is a closed comment "-->" alone at end of tag false --> On Tue, Jun 30, 2015 at 2:09 PM, ssharma7...@gmail.com < ssharma7...@gmail.com> wrote: > Vincenzo D'Amore, > The following is my (CURRENT) Working Final Configuration: > > *Scheme.xml* > > . > . > termVectors="true" termPositions="true" termOffsets="true" /> > stored="true" required="true" multiValued="false" /> > . > . > > > > . > . > > positionIncrementGap="100"> > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.RemoveDuplicatesTokenFilterFactory"/> > > class="solr.LowerCaseFilterFactory"/> > > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.LowerCaseFilterFactory"/> > > > > positionIncrementGap="100"> > > class="solr.KeywordTokenizerFactory"/> > class="solr.LowerCaseFilterFactory"/> > > > class="solr.KeywordTokenizerFactory"/> > class="solr.LowerCaseFilterFactory"/> > > > > . > . > > > > *solrconfig.xml* > .. > .. > > > textSuggester > FreeTextLookupFactory > DocumentDictionaryFactory > text > c_text > true > > > docNameSuggester > FreeTextLookupFactory > DocumentDictionaryFactory > document_name > c_document_name > true > > > > startup="lazy" > > > json > true > 5 > false--> > > textSuggester > docNameSuggester > > > suggest > > > .. > .. > > *Solr Query URL* > > http://localhost:8983/solr/collection1/suggestHandler?&wt=xml&suggest.q=document > > *Suggester Output* > > > > > 0 > 16 > > > > > 5 > > > document > 512409557603043072 > > > > document1 > 512409557603043072 > > > > document2 > 512409557603043072 > > > > document3 > 512409557603043072 > > > > document4 > 512409557603043072 > > > > > > > > 3 > > > document > 10933347601771902 > > > > documents > 4373339040708760 > > > > documenting > 2186669520354380 > > > > > > > > > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086p4214929.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
Re: Solr Suggester not working.
Vincenzo D'Amore, The following is my (CURRENT) Working Final Configuration: *Scheme.xml* . . . . . . . . *solrconfig.xml* .. .. textSuggester FreeTextLookupFactory DocumentDictionaryFactory text c_text true docNameSuggester FreeTextLookupFactory DocumentDictionaryFactory document_name c_document_name true json true 5 false--> textSuggester docNameSuggester suggest .. .. *Solr Query URL* http://localhost:8983/solr/collection1/suggestHandler?&wt=xml&suggest.q=document *Suggester Output* 0 16 5 document 512409557603043072 document1 512409557603043072 document2 512409557603043072 document3 512409557603043072 document4 512409557603043072 3 document 10933347601771902 documents 4373339040708760 documenting 2186669520354380 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086p4214929.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester not working.
Hi, can you post your final configuration? On Tue, Jun 30, 2015 at 9:57 AM, ssharma7...@gmail.com < ssharma7...@gmail.com> wrote: > davidphilip cherian & Alessandro Benedetti, > Thanks for you feedback & links, I was able to get the suggestions from > suggester component. > > Thanks & Regards, > Sachin Vyas. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086p4214873.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
Re: Solr Suggester not working.
davidphilip cherian & Alessandro Benedetti, Thanks for you feedback & links, I was able to get the suggestions from suggester component. Thanks & Regards, Sachin Vyas. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086p4214873.html Sent from the Solr - User mailing list archive at Nabble.com.
solr suggester build issues
Solr : 4.9.x , with simple solr cloud on jetty.
JDK 1.7
num of replica : 4 , one replica for each shard
num of shards : 1
Hi All,
I have been facing below issues with solr suggester introduced in 4.7.x. Do
any one have good working solution or
buildOnCommit=true property is suggested not to use with index with more
frequent softcommits as suggested in the documentation
https://cwiki.apache.org/confluence/display/solr/Suggester
So we have disabled this (buildOnCommit=false) and started using
buildOnOptimize=true, which was not helping us to have latest document
suggestion (with frequent softcommits),
as hardly there was one optimize each day. (we have default optimize
setting in solrconfig)
So we have disabled buildOnOptimize (buildOnOptimize=false)
As suggested in the documentation, as of now, we came up with cron jobs to
build the suggester for every hour.
These jobs are doing their job, i.e, we are having the latest suggestions
available every hour, below are issues that we have this implementation.
*Issue#1* : Suggest built url i.e,
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true* if
issued to one replica of solr cloud does not build suggesters in all of the
replicas in solrcloud.
Resolution: For which we have separate cron jobs on each of the
solr instance having the build call to build the suggester, below is the
raw pictorial representation of this impl
(which is not the best implementation which has
many flaws)
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true*
* |*
* |--
suggestcron.job.sh <http://suggestcron.job.sh> (on solr1.aws.instance)*
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true*
* |*
* |--
suggestcron.job.sh <http://suggestcron.job.sh> (on solr2.aws.instance)*
* .. similar for other solr nodes*
* We will be coming up with single script to go this for all
collection later.*
we were bit happy that we are having a updated suggester in all of the
instances, *which is not!*
*The issue#2 the suggester built on all solr nodes were not consistent as
the solr core in each solr replica have difference in max-docs and
num-docs *
*(which is quiet normal **with frequent softcommits , when updates mostly
have the same documents updated with different data, **i guess , correct me
if i'm wrong )*
when we query curl -i "http://
$solrnode:8983/solr/liveaodfuture/suggest?q=Nirvana&wt=json&indent=true"
one of the solr node returns
{
"responseHeader":{
"status":0,
"QTime":0},
"suggest":{
"AnalyzingSuggester":{
"Nirvana":{
"numFound":1,
"suggestions":[{
"term":"nirvana",
"weight":6,
"payload":""}]}},
"DictionarySuggester":{
"Nirvana":{
"numFound":0,
"suggestions":[]
/admin/luke/collection/ call status
"index":{
"numDocs":90564,
"maxDoc":94583,
"deletedDocs":4019,
...}
while other 3 solr node returns
{
"responseHeader":{
"status":0,
"QTime":1},
"suggest":{
"AnalyzingSuggester":{
"Nirvana":{
"numFound":2,
"suggestions":[{
"term":"nirvana",
"weight":163,
"payload":""},
* {*
*"term":"nirvana cover",*
*"weight":11,*
*"payload":""}]}},*
"DictionarySuggester":{
"Nirvana":{
"numFound":0,
"suggestions":[]
/admin/luke/collection/ call status on other 3 solr nodes... which have
different maxDoc that the above solr node.
"index":{
"numDocs":90564,
"maxDoc":156760,
}
when i check the built time for suggest directory of the collection on each
solr node have the same time
ls -lah /mnt/solrdrive/solr/cores/*/data/suggest_analyzing/*
-rw-r--r-- 1 root root 3.0M May 20 16:00
/mnt/solrdrive/solr/cores/collection1_shard1_replica3/data/suggest_analyzing/wfsta.bin
Questions:
Does the suggester built url i.e,
*http://$solrnode:8983/solr/collection1/suggest?suggest.build=true
*consider maxdocs or deleted docs also?
Does the suggester built from i.e,
*solr/collection1/suggest?suggest.build=true
*is different from buildOnCommit=true property ?
Do any one have better solution to keep the suggester current
with contents in the index with more frequent softcommits?
Does solr have any component like scheduler like cron scheduler
to schedule the suggest build and
scheduling the optimize on daily basis ?
*Thanks,*
*Rajesh**.*
Re: Solr Suggester not working.
I agree with David, I see a Ton of wrong configuration in yours … Please have a read of the documentation linked . And take a look to this mailing list, we have tons of messages related that can help you. A first suggestion for you anyway, is to take care of your analysis. The suggestions will be the tokens you produce with the analyser you provide to the Suggester, so make extra care of them. I see this : text string Which does not make a lot of sense … Cheers 2015-06-26 7:35 GMT+01:00 ssharma7...@gmail.com : > Hi, > I am new to Solr & have installed Solr 5.1. I am trying to get the > Suggester > component working without any success, the following are the relevant > details: > > *Schema.xml* > termVectors="true" termPositions="true" termOffsets="true" /> > > positionIncrementGap="100"> > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.RemoveDuplicatesTokenFilterFactory"/> > > class="solr.LowerCaseFilterFactory"/> > > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.LowerCaseFilterFactory"/> > > > > *solr-config.xml* > > > > textSuggester > FuzzyLookupFactory > DocumentDictionaryFactory > text > string > true > > > > startup="lazy" > > > true > 10 > textSuggester > > > suggest > > > > *Query to get suggestions* > > http://localhost:8983/solr/portal_documents/suggest?suggest=true&suggest.dictionary=textSuggester&suggest.q=testing > > *Above Query Output* > > > 0 > 31 > > > > > 0 > > > > > > > The data is getting indexed properly, as can be seen from image below: > <http://lucene.472066.n3.nabble.com/file/n4214086/solr-suggester.jpg> > > Regards, > Sachin Vyas. > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086.html > Sent from the Solr - User mailing list archive at Nabble.com. > -- -- Benedetti Alessandro Visiting card : http://about.me/alessandro_benedetti "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
Re: Solr Suggester not working.
Did you have a quick look at this blog? http://lucidworks.com/blog/solr-suggester/ On Fri, Jun 26, 2015 at 12:05 PM, ssharma7...@gmail.com < ssharma7...@gmail.com> wrote: > Hi, > I am new to Solr & have installed Solr 5.1. I am trying to get the > Suggester > component working without any success, the following are the relevant > details: > > *Schema.xml* > termVectors="true" termPositions="true" termOffsets="true" /> > > positionIncrementGap="100"> > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.RemoveDuplicatesTokenFilterFactory"/> > > class="solr.LowerCaseFilterFactory"/> > > > class="solr.UAX29URLEmailTokenizerFactory"/> > ignoreCase="true" > words="lang/stopwords_en.txt" /> > class="solr.ASCIIFoldingFilterFactory"/> > class="solr.EnglishPossessiveFilterFactory"/> > class="solr.LowerCaseFilterFactory"/> > > > > *solr-config.xml* > > > > textSuggester > FuzzyLookupFactory > DocumentDictionaryFactory > text > string > true > > > > startup="lazy" > > > true > 10 > textSuggester > > > suggest > > > > *Query to get suggestions* > > http://localhost:8983/solr/portal_documents/suggest?suggest=true&suggest.dictionary=textSuggester&suggest.q=testing > > *Above Query Output* > > > 0 > 31 > > > > > 0 > > > > > > > The data is getting indexed properly, as can be seen from image below: > <http://lucene.472066.n3.nabble.com/file/n4214086/solr-suggester.jpg> > > Regards, > Sachin Vyas. > > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Solr Suggester not working.
Hi, I am new to Solr & have installed Solr 5.1. I am trying to get the Suggester component working without any success, the following are the relevant details: *Schema.xml* *solr-config.xml* textSuggester FuzzyLookupFactory DocumentDictionaryFactory text string true true 10 textSuggester suggest *Query to get suggestions* http://localhost:8983/solr/portal_documents/suggest?suggest=true&suggest.dictionary=textSuggester&suggest.q=testing *Above Query Output* 0 31 0 The data is getting indexed properly, as can be seen from image below: <http://lucene.472066.n3.nabble.com/file/n4214086/solr-suggester.jpg> Regards, Sachin Vyas. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-not-working-tp4214086.html Sent from the Solr - User mailing list archive at Nabble.com.
AW: Occasionally getting error in solr suggester component.
Thanks! -Ursprüngliche Nachricht- Von: Erick Erickson [mailto:erickerick...@gmail.com] Gesendet: Montag, 1. Juni 2015 17:26 An: solr-user@lucene.apache.org Betreff: Re: Occasionally getting error in solr suggester component. Attach suggester.build=true or suggester.buildAll=true to any request to suggester to rebuild. OR add buildOnStartup or buildOnCommit or buildOnOptimize to the definition in solrConfig. BUT: building can be a _very_ expensive operation. For document-based indexes, the build process reads through _all_ of the _stored_ documents in your index, and that can take many minutes so I recommend against these options for a large index, and strongly recommend you test these with a large corpus. Best, Erick On Mon, Jun 1, 2015 at 4:01 AM, Clemens Wyss DEV wrote: > Lucene 5.1: > I am (also) facing > "java.lang.IllegalStateException: suggester was not built" > > At the very moment no new documents seem tob e added to the index/core. Will > a reboot "sanitize" the index/core? > > I (still) have > name="buildOnCommit">true > > How can I tell Solr to peridoically update the suggestions? If not possible > per configuration (in solrconfig.xml), what ist he preferred approach through > SolrJ? > > Thx > Clemens > > -Ursprüngliche Nachricht- > Von: Michael Sokolov [mailto:msoko...@safaribooksonline.com] > Gesendet: Donnerstag, 15. Januar 2015 19:52 > An: solr-user@lucene.apache.org > Betreff: Re: Occasionally getting error in solr suggester component. > > That sounds like a good approach to me. Of course it depends how often you > commit, and what your tolerance is for delay in having suggestions appear, > but it sounds as if you have a good understanding of the tradeoffs there. > > -Mike > > On 1/15/15 10:31 AM, Dhanesh Radhakrishnan wrote: >> Hi, >> From Solr 4.7 onwards, the implementation of this Suggester is >> changed. The old SpellChecker based search component is replaced with >> a new suggester that utilizes Lucene suggester module. The latest >> Solr download is preconfigured with this new suggester I;m using Solr >> 4.10 and suggestion are based on query /suggest instead of /spell. >> So what I did is that in changed to > name="buildOnCommit">false Its not good that each time rebuild >> the index on commit , however, I would like to build the index on >> certain time period, say 1 hour. >> The lookup data will be built only when requested by URL parameter >> suggest.build=true >> >> "http://localhost:8983/solr/ha/suggest?suggest.build=true"; >> >> So this will rebuild the index again and the changes will reflect in >> the suggester. >> >> There are certain pros and cons for this. >> Issue is that the change will reflect only on certain time interval, >> here 1 hour. Advantage is that we can avoid the rebuilt index on >> every commit or optimize. >> >> Is this the right way ?? or any that I missed ??? >> >> Regards >> dhanesh s.r >> >> >> >> >> On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov < >> msoko...@safaribooksonline.com> wrote: >> >>> did you build the spellcheck index using spellcheck.build as >>> described >>> here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ? >>> >>> -Mike >>> >>> >>> On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote: >>> >>>> Hi, >>>> Thanks for the reply. >>>> As you mentioned in the previous mail I changed buildOnCommit=false >>>> in solrConfig. >>>> After that change, suggestions are not working. >>>> In Solr 4.7 introduced a new approach based on a dedicated >>>> SuggestComponent I'm using that component to build suggestions and >>>> lookup implementation is "AnalyzingInfixLookupFactory" >>>> Is there any work around ?? >>>> >>>> >>>> >>>> >>>> On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov < >>>> msoko...@safaribooksonline.com> wrote: >>>> >>>> I think you are probably getting bitten by one of the issues >>>> addressed in >>>>> LUCENE-5889 >>>>> >>>>> I would recommend against using buildOnCommit=true - with a large >>>>> index this can be a performance-killer. Instead, build the index >>>>> yourself using the Solr spellchecker support >>>>> (spellcheck.build=true) >>>>> >>>>> -Mike &g
Re: Occasionally getting error in solr suggester component.
Attach suggester.build=true or suggester.buildAll=true to any request to suggester to rebuild. OR add buildOnStartup or buildOnCommit or buildOnOptimize to the definition in solrConfig. BUT: building can be a _very_ expensive operation. For document-based indexes, the build process reads through _all_ of the _stored_ documents in your index, and that can take many minutes so I recommend against these options for a large index, and strongly recommend you test these with a large corpus. Best, Erick On Mon, Jun 1, 2015 at 4:01 AM, Clemens Wyss DEV wrote: > Lucene 5.1: > I am (also) facing > "java.lang.IllegalStateException: suggester was not built" > > At the very moment no new documents seem tob e added to the index/core. Will > a reboot "sanitize" the index/core? > > I (still) have > name="buildOnCommit">true > > How can I tell Solr to peridoically update the suggestions? If not possible > per configuration (in solrconfig.xml), what ist he preferred approach through > SolrJ? > > Thx > Clemens > > -Ursprüngliche Nachricht- > Von: Michael Sokolov [mailto:msoko...@safaribooksonline.com] > Gesendet: Donnerstag, 15. Januar 2015 19:52 > An: solr-user@lucene.apache.org > Betreff: Re: Occasionally getting error in solr suggester component. > > That sounds like a good approach to me. Of course it depends how often you > commit, and what your tolerance is for delay in having suggestions appear, > but it sounds as if you have a good understanding of the tradeoffs there. > > -Mike > > On 1/15/15 10:31 AM, Dhanesh Radhakrishnan wrote: >> Hi, >> From Solr 4.7 onwards, the implementation of this Suggester is >> changed. The old SpellChecker based search component is replaced with >> a new suggester that utilizes Lucene suggester module. The latest Solr >> download is preconfigured with this new suggester I;m using Solr 4.10 >> and suggestion are based on query /suggest instead of /spell. >> So what I did is that in changed to > name="buildOnCommit">false Its not good that each time rebuild >> the index on commit , however, I would like to build the index on >> certain time period, say 1 hour. >> The lookup data will be built only when requested by URL parameter >> suggest.build=true >> >> "http://localhost:8983/solr/ha/suggest?suggest.build=true"; >> >> So this will rebuild the index again and the changes will reflect in >> the suggester. >> >> There are certain pros and cons for this. >> Issue is that the change will reflect only on certain time interval, >> here 1 hour. Advantage is that we can avoid the rebuilt index on >> every commit or optimize. >> >> Is this the right way ?? or any that I missed ??? >> >> Regards >> dhanesh s.r >> >> >> >> >> On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov < >> msoko...@safaribooksonline.com> wrote: >> >>> did you build the spellcheck index using spellcheck.build as >>> described >>> here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ? >>> >>> -Mike >>> >>> >>> On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote: >>> >>>> Hi, >>>> Thanks for the reply. >>>> As you mentioned in the previous mail I changed buildOnCommit=false >>>> in solrConfig. >>>> After that change, suggestions are not working. >>>> In Solr 4.7 introduced a new approach based on a dedicated >>>> SuggestComponent I'm using that component to build suggestions and >>>> lookup implementation is "AnalyzingInfixLookupFactory" >>>> Is there any work around ?? >>>> >>>> >>>> >>>> >>>> On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov < >>>> msoko...@safaribooksonline.com> wrote: >>>> >>>> I think you are probably getting bitten by one of the issues >>>> addressed in >>>>> LUCENE-5889 >>>>> >>>>> I would recommend against using buildOnCommit=true - with a large >>>>> index this can be a performance-killer. Instead, build the index >>>>> yourself using the Solr spellchecker support >>>>> (spellcheck.build=true) >>>>> >>>>> -Mike >>>>> >>>>> >>>>> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote: >>>>> >>>>> Hi all, >>>>>> I am experiencing a problem in Solr SuggestComponent Occasionally >>>
AW: Occasionally getting error in solr suggester component.
Lucene 5.1:
I am (also) facing
"java.lang.IllegalStateException: suggester was not built"
At the very moment no new documents seem tob e added to the index/core. Will a
reboot "sanitize" the index/core?
I (still) have
name="buildOnCommit">true
How can I tell Solr to peridoically update the suggestions? If not possible per
configuration (in solrconfig.xml), what ist he preferred approach through SolrJ?
Thx
Clemens
-Ursprüngliche Nachricht-
Von: Michael Sokolov [mailto:msoko...@safaribooksonline.com]
Gesendet: Donnerstag, 15. Januar 2015 19:52
An: solr-user@lucene.apache.org
Betreff: Re: Occasionally getting error in solr suggester component.
That sounds like a good approach to me. Of course it depends how often you
commit, and what your tolerance is for delay in having suggestions appear, but
it sounds as if you have a good understanding of the tradeoffs there.
-Mike
On 1/15/15 10:31 AM, Dhanesh Radhakrishnan wrote:
> Hi,
> From Solr 4.7 onwards, the implementation of this Suggester is
> changed. The old SpellChecker based search component is replaced with
> a new suggester that utilizes Lucene suggester module. The latest Solr
> download is preconfigured with this new suggester I;m using Solr 4.10
> and suggestion are based on query /suggest instead of /spell.
> So what I did is that in changed to name="buildOnCommit">false Its not good that each time rebuild
> the index on commit , however, I would like to build the index on
> certain time period, say 1 hour.
> The lookup data will be built only when requested by URL parameter
> suggest.build=true
>
> "http://localhost:8983/solr/ha/suggest?suggest.build=true";
>
> So this will rebuild the index again and the changes will reflect in
> the suggester.
>
> There are certain pros and cons for this.
> Issue is that the change will reflect only on certain time interval,
> here 1 hour. Advantage is that we can avoid the rebuilt index on
> every commit or optimize.
>
> Is this the right way ?? or any that I missed ???
>
> Regards
> dhanesh s.r
>
>
>
>
> On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov <
> msoko...@safaribooksonline.com> wrote:
>
>> did you build the spellcheck index using spellcheck.build as
>> described
>> here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ?
>>
>> -Mike
>>
>>
>> On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote:
>>
>>> Hi,
>>> Thanks for the reply.
>>> As you mentioned in the previous mail I changed buildOnCommit=false
>>> in solrConfig.
>>> After that change, suggestions are not working.
>>> In Solr 4.7 introduced a new approach based on a dedicated
>>> SuggestComponent I'm using that component to build suggestions and
>>> lookup implementation is "AnalyzingInfixLookupFactory"
>>> Is there any work around ??
>>>
>>>
>>>
>>>
>>> On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov <
>>> msoko...@safaribooksonline.com> wrote:
>>>
>>> I think you are probably getting bitten by one of the issues
>>> addressed in
>>>> LUCENE-5889
>>>>
>>>> I would recommend against using buildOnCommit=true - with a large
>>>> index this can be a performance-killer. Instead, build the index
>>>> yourself using the Solr spellchecker support
>>>> (spellcheck.build=true)
>>>>
>>>> -Mike
>>>>
>>>>
>>>> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
>>>>
>>>> Hi all,
>>>>> I am experiencing a problem in Solr SuggestComponent Occasionally
>>>>> solr suggester component throws an error like
>>>>>
>>>>> Solr failed:
>>>>> {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggest
>>>>> er
>>>>> was
>>>>> not built","trace":"java.lang.IllegalStateException: suggester was
>>>>> not built\n\tat
>>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>>> lookup(AnalyzingInfixSuggester.java:368)\n\tat
>>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>>> lookup(AnalyzingInfixSuggester.java:342)\n\tat
>>>>> org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\
>>>>> tat org.apache.solr.spelling.suggest.SolrSuggester.
>&g
Re: Solr suggester
right. File-based suggestions should be much faster to build, but it's certainly the case with large indexes that you have to build it periodically so they won't be completely up to date. However, this stuff is way cool. AnalyzingInfixSuggester, for instance, suggests entire fields rather than isolated words, returning the original case, punctuation etc. The index-based spellcheck/suggest just reads terms from the indexed fields which takes no time to build but suffers from reading _indexed_ terms, i.e. terms that have gone through the analysis process that may have been stemmed, lowercased, all that. On Thu, May 21, 2015 at 9:03 AM, jon kerling wrote: > Hi Erick, > I have read your blog and it is really helpful.I'm thinking about upgrading > to Solr 5.1 but it won't solve all my problems with this issue, as you said > each build will have to read all docs, and analyze it's fields. The only > advantage is that I can skip default suggest.build on start up. > Thank you for your reply. > Jon Kerling. > > > > On Thursday, May 21, 2015 6:38 PM, Erick Erickson > wrote: > > > Frankly, the suggester is rather broken in Solr 4.x with large > indexes. Building the suggester index (or FST) requires that _all_ the > docs get read, the stored fields analyzed and added to the suggester. > Unfortunately, this happens _every_ time you start Solr and can take > many minutes whether or not you have buildOnStartup set to false, see: > https://issues.apache.org/jira/browse/SOLR-6845. > > See: http://lucidworks.com/blog/solr-suggester/ > > See inline. > > On Thu, May 21, 2015 at 6:12 AM, jon kerling > wrote: >> Hi, >> >> I'm using solr 4.10 and I'm trying to add autosuggest ability to my >> application. >> I'm currently using this kind of configuration: >> >> >> >> mySuggester >> FuzzyLookupFactory >> suggester_fuzzy_dir >> DocumentDictionaryFactory >> field2 >> weightField >> text_general >> >> >> >> >> >> true >> 10 >> mySuggester >> >> >> suggest >> >> >> >> I wanted to know how the suggester Index/file is being rebuilt. >> Is it suppose to have all the terms of the desired field in the suggester? > Yes. >> if not, is it related to this kind of lookup implementation? >> if I'll use other lookup implementation which suggest also infix terms of >> fields, >> doesn't it has to hold all terms of the field? > Yes. >> >> When i call suggest.build, does it build from scratch the suggester >> Index/file, >> or is it just doing something like sort of "delta" indexing suggestions? > Builds from scratch >> >> Thank You, >> Jon > > >
Re: Solr suggester
Hi Erick, I have read your blog and it is really helpful.I'm thinking about upgrading to Solr 5.1 but it won't solve all my problems with this issue, as you said each build will have to read all docs, and analyze it's fields. The only advantage is that I can skip default suggest.build on start up. Thank you for your reply. Jon Kerling. On Thursday, May 21, 2015 6:38 PM, Erick Erickson wrote: Frankly, the suggester is rather broken in Solr 4.x with large indexes. Building the suggester index (or FST) requires that _all_ the docs get read, the stored fields analyzed and added to the suggester. Unfortunately, this happens _every_ time you start Solr and can take many minutes whether or not you have buildOnStartup set to false, see: https://issues.apache.org/jira/browse/SOLR-6845. See: http://lucidworks.com/blog/solr-suggester/ See inline. On Thu, May 21, 2015 at 6:12 AM, jon kerling wrote: > Hi, > > I'm using solr 4.10 and I'm trying to add autosuggest ability to my > application. > I'm currently using this kind of configuration: > > > > mySuggester > FuzzyLookupFactory > suggester_fuzzy_dir > DocumentDictionaryFactory > field2 > weightField > text_general > > > > > > true > 10 > mySuggester > > > suggest > > > > I wanted to know how the suggester Index/file is being rebuilt. > Is it suppose to have all the terms of the desired field in the suggester? Yes. > if not, is it related to this kind of lookup implementation? > if I'll use other lookup implementation which suggest also infix terms of > fields, > doesn't it has to hold all terms of the field? Yes. > > When i call suggest.build, does it build from scratch the suggester > Index/file, > or is it just doing something like sort of "delta" indexing suggestions? Builds from scratch > > Thank You, > Jon
Re: Solr suggester
Frankly, the suggester is rather broken in Solr 4.x with large indexes. Building the suggester index (or FST) requires that _all_ the docs get read, the stored fields analyzed and added to the suggester. Unfortunately, this happens _every_ time you start Solr and can take many minutes whether or not you have buildOnStartup set to false, see: https://issues.apache.org/jira/browse/SOLR-6845. See: http://lucidworks.com/blog/solr-suggester/ See inline. On Thu, May 21, 2015 at 6:12 AM, jon kerling wrote: > Hi, > > I'm using solr 4.10 and I'm trying to add autosuggest ability to my > application. > I'm currently using this kind of configuration: > > > > mySuggester > FuzzyLookupFactory > suggester_fuzzy_dir > DocumentDictionaryFactory > field2 > weightField > text_general > > > > > > true > 10 > mySuggester > > > suggest > > > > I wanted to know how the suggester Index/file is being rebuilt. > Is it suppose to have all the terms of the desired field in the suggester? Yes. > if not, is it related to this kind of lookup implementation? > if I'll use other lookup implementation which suggest also infix terms of > fields, > doesn't it has to hold all terms of the field? Yes. > > When i call suggest.build, does it build from scratch the suggester > Index/file, > or is it just doing something like sort of "delta" indexing suggestions? Builds from scratch > > Thank You, > Jon
Solr suggester
Hi, I'm using solr 4.10 and I'm trying to add autosuggest ability to my application. I'm currently using this kind of configuration: mySuggester FuzzyLookupFactory suggester_fuzzy_dir DocumentDictionaryFactory field2 weightField text_general true 10 mySuggester suggest I wanted to know how the suggester Index/file is being rebuilt. Is it suppose to have all the terms of the desired field in the suggester? if not, is it related to this kind of lookup implementation? if I'll use other lookup implementation which suggest also infix terms of fields, doesn't it has to hold all terms of the field? When i call suggest.build, does it build from scratch the suggester Index/file, or is it just doing something like sort of "delta" indexing suggestions? Thank You, Jon
Re: Issue with Solr Suggester
This is working as expected but the problem I get with wrong spelling searches. When I give suggest.q=Bhopml Fuzzy lookup suggests bhopal which is correct buy in AnalyzingInfixSuggester it doesn't provide this . Regards, Swaraj Kumar Senior Software Engineer I MakeMyTrip.com Mob No- 9811774497 On Thu, Apr 23, 2015 at 9:38 PM, Erick Erickson wrote: > I'm pretty sure that FuzzyLookup only goes from the beginning of the > field, so this is not surprising. To get what you're looking for you > probably would get more joy from the AnalyzingInfixSuggester. > > Best, > Erick > > On Thu, Apr 23, 2015 at 6:20 AM, Swaraj Kumar > wrote: > > I am trying to implement Suggester in SOLR 5.0, > > > > Below is my configuration :- > > > > > > my-suggester > > FuzzyLookupFactory > > DocumentDictionaryFactory > > name > > cityname > > id > > text_general > > true > > > > > > > >startup="lazy"> > > > > true > > 10 > > > > > > suggest > > > > > > > > > > In Schema.xml :- > > > > > > > positionIncrementGap="100"> > > > > > > > words="stopwords.txt" /> > > > > > > > > > > > words="stopwords.txt" /> > > > ignoreCase="true" expand="true"/> > > > > > > > > > > My data is like :- > > > > Delhi > > New Delhi > > The New Castle > > > > > > > > When I Query using following parameter below :- > > > http://localhost/solr/location/suggest?suggest.dictionary=my-suggester&suggest=true&suggest.build=true&suggest.q=Delhi > > > > I get "Delhi" only in result but the ideal result is both Delhi and New > > Delhi. > > > > > > It will be very helpful if you guyz suggest me how to achieve this. > > > > > > Regards, > > Swaraj >
Re: Issue with Solr Suggester
I'm pretty sure that FuzzyLookup only goes from the beginning of the field, so this is not surprising. To get what you're looking for you probably would get more joy from the AnalyzingInfixSuggester. Best, Erick On Thu, Apr 23, 2015 at 6:20 AM, Swaraj Kumar wrote: > I am trying to implement Suggester in SOLR 5.0, > > Below is my configuration :- > > > my-suggester > FuzzyLookupFactory > DocumentDictionaryFactory > name > cityname > id > text_general > true > > > > > > true > 10 > > > suggest > > > > > In Schema.xml :- > > > positionIncrementGap="100"> > > > words="stopwords.txt" /> > > > > > words="stopwords.txt" /> > ignoreCase="true" expand="true"/> > > > > > My data is like :- > > Delhi > New Delhi > The New Castle > > > > When I Query using following parameter below :- > http://localhost/solr/location/suggest?suggest.dictionary=my-suggester&suggest=true&suggest.build=true&suggest.q=Delhi > > I get "Delhi" only in result but the ideal result is both Delhi and New > Delhi. > > > It will be very helpful if you guyz suggest me how to achieve this. > > > Regards, > Swaraj
Issue with Solr Suggester
I am trying to implement Suggester in SOLR 5.0, Below is my configuration :- my-suggester FuzzyLookupFactory DocumentDictionaryFactory name cityname id text_general true true 10 suggest In Schema.xml :- My data is like :- Delhi New Delhi The New Castle When I Query using following parameter below :- http://localhost/solr/location/suggest?suggest.dictionary=my-suggester&suggest=true&suggest.build=true&suggest.q=Delhi I get "Delhi" only in result but the ideal result is both Delhi and New Delhi. It will be very helpful if you guyz suggest me how to achieve this. Regards, Swaraj
Re: Solr Suggester Autocomplete Working Example
Alexandre Rafalovitch wrote > Actually, you have a capability to do unbelievable level of > customization in Solr, starting from schema definition and down to > writing custom components in Java. Or even completely rebuilding Solr > the way you want from sources. Or was that a reference to your current > skills rather than than Solr's? I think that should be fixable as > well. Just keep learning and asking questions. We'll try to help. > > As to the suggester, it may make sense to explain what kind of text > you are providing and what results you might be expecting. A bit more > details that you've given already. There are several different > implementations, each with its own trade-offs. > > Regards, >Alex. Sorry Alex, I am just a bit dumb. That reference was regarding my skills not Solr's. I think Michael pointed out one of my problems i.e. I was using the Search field that had been stemmed. I will look at creating an alternate field just for the suggester. Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-Autocomplete-Working-Example-tp4183493p4183532.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester Autocomplete Working Example
Thank you Michael. I will look at safaribooksonline.com later today when I create my account. I am not sure how to use AnalyzingInfixSuggester. I googled a bit, and I can find the source code, but not how to use it. You are perfectly correct when you say that I am using a field also used for searching and which has been stemmed. I need to look into setting up another field for the suggester. I will post here when I have questions about this. Thanks again. Michael Sokolov-3 wrote > Please go ahead and play with autocomplete on safaribooksonline.com/home > - if you are not a subscriber you will have to sign up for a free > trial. We use the AnalyzingInfixSuggester. From your description, it > sounds as if you are building completions from a field that you also use > for searching -- maybe it is stemmed, and that's why you are seeing the > weird partial words. To get good results from the suggester you will > probably need to set up a special field to use as a source of > suggestions that uses appropriate text analysis. > > -Mike -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-Autocomplete-Working-Example-tp4183493p4183530.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Suggester Autocomplete Working Example
On 2 February 2015 at 11:26, O. Olson wrote: > I also know that I do not have the > capability to do a lot of customizations to Solr that are much beyond the > defaults and changing a few settings. Actually, you have a capability to do unbelievable level of customization in Solr, starting from schema definition and down to writing custom components in Java. Or even completely rebuilding Solr the way you want from sources. Or was that a reference to your current skills rather than than Solr's? I think that should be fixable as well. Just keep learning and asking questions. We'll try to help. As to the suggester, it may make sense to explain what kind of text you are providing and what results you might be expecting. A bit more details that you've given already. There are several different implementations, each with its own trade-offs. Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/
Re: Solr Suggester Autocomplete Working Example
Please go ahead and play with autocomplete on safaribooksonline.com/home - if you are not a subscriber you will have to sign up for a free trial. We use the AnalyzingInfixSuggester. From your description, it sounds as if you are building completions from a field that you also use for searching -- maybe it is stemmed, and that's why you are seeing the weird partial words. To get good results from the suggester you will probably need to set up a special field to use as a source of suggestions that uses appropriate text analysis. -Mike On 2/2/2015 11:26 AM, O. Olson wrote: Hi, I'm am wondering if anyone can point me to a website that user Solr's Suggester or Autocomplete or whatever you call it. I am looking for something that is closer to the default provided in the examples, but is also used commercially. I have a local Solr installation that is on an intranet. (Sorry I cannot post it here.) Unfortunately, the suggestions it provides does not seem to be OK. By this I mean in comparison to Google, which I know does not use Solr. For example, when I type the string "sto" into my installation, I get the suggested values like "storag" - which is not a complete word i.e. it misses the 'e' in the end. On the other hand when I use Google, I get complete words like "stock market" or "stopwatch" etc. I know Google does not use Solr. I also know that I do not have the capability to do a lot of customizations to Solr that are much beyond the defaults and changing a few settings. Hence I am curious if there is a website out there that uses Suggester or Autocomplete where I can compare the capabilities with my own. Thank you -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-Autocomplete-Working-Example-tp4183493.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr Suggester Autocomplete Working Example
Hi, I'm am wondering if anyone can point me to a website that user Solr's Suggester or Autocomplete or whatever you call it. I am looking for something that is closer to the default provided in the examples, but is also used commercially. I have a local Solr installation that is on an intranet. (Sorry I cannot post it here.) Unfortunately, the suggestions it provides does not seem to be OK. By this I mean in comparison to Google, which I know does not use Solr. For example, when I type the string "sto" into my installation, I get the suggested values like "storag" - which is not a complete word i.e. it misses the 'e' in the end. On the other hand when I use Google, I get complete words like "stock market" or "stopwatch" etc. I know Google does not use Solr. I also know that I do not have the capability to do a lot of customizations to Solr that are much beyond the defaults and changing a few settings. Hence I am curious if there is a website out there that uses Suggester or Autocomplete where I can compare the capabilities with my own. Thank you -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Suggester-Autocomplete-Working-Example-tp4183493.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Occasionally getting error in solr suggester component.
That sounds like a good approach to me. Of course it depends how often
you commit, and what your tolerance is for delay in having suggestions
appear, but it sounds as if you have a good understanding of the
tradeoffs there.
-Mike
On 1/15/15 10:31 AM, Dhanesh Radhakrishnan wrote:
Hi,
From Solr 4.7 onwards, the implementation of this Suggester is changed. The
old SpellChecker based search component is replaced with a new suggester
that utilizes Lucene suggester module. The latest Solr download is
preconfigured with this new suggester
I;m using Solr 4.10 and suggestion are based on query /suggest instead of
/spell.
So what I did is that in changed to false
Its not good that each time rebuild the index on commit , however, I
would like to build
the index on certain time period, say 1 hour.
The lookup data will be built only when requested by URL parameter
suggest.build=true
"http://localhost:8983/solr/ha/suggest?suggest.build=true";
So this will rebuild the index again and the changes will reflect in the
suggester.
There are certain pros and cons for this.
Issue is that the change will reflect only on certain time interval, here 1
hour. Advantage is that we can avoid the rebuilt index on every commit or
optimize.
Is this the right way ?? or any that I missed ???
Regards
dhanesh s.r
On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
did you build the spellcheck index using spellcheck.build as described
here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ?
-Mike
On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote:
Hi,
Thanks for the reply.
As you mentioned in the previous mail I changed buildOnCommit=false in
solrConfig.
After that change, suggestions are not working.
In Solr 4.7 introduced a new approach based on a dedicated
SuggestComponent
I'm using that component to build suggestions and lookup implementation is
"AnalyzingInfixLookupFactory"
Is there any work around ??
On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
I think you are probably getting bitten by one of the issues addressed in
LUCENE-5889
I would recommend against using buildOnCommit=true - with a large index
this can be a performance-killer. Instead, build the index yourself
using
the Solr spellchecker support (spellcheck.build=true)
-Mike
On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
Hi all,
I am experiencing a problem in Solr SuggestComponent
Occasionally solr suggester component throws an error like
Solr failed:
{"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester
was
not built","trace":"java.lang.IllegalStateException: suggester was not
built\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
lookup(AnalyzingInfixSuggester.java:368)\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
lookup(AnalyzingInfixSuggester.java:342)\n\tat
org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
org.apache.solr.spelling.suggest.SolrSuggester.
getSuggestions(SolrSuggester.java:199)\n\tat
org.apache.solr.handler.component.SuggestComponent.
process(SuggestComponent.java:234)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(
SearchHandler.java:218)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(
RequestHandlerBase.java:135)\n\tat
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
handleRequest(RequestHandlers.java:246)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(
SolrDispatchFilter.java:777)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(
SolrDispatchFilter.java:418)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(
SolrDispatchFilter.java:207)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:243)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:210)\n\tat
org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:225)\n\tat
org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:123)\n\tat
org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:168)\n\tat
org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:98)\n\tat
org.apache.catalina.valves.AccessLogValve.invoke(
AccessLogValve.java:927)\n\tat
org.apache.catalina.valves.RemoteIpValve.invoke(
RemoteIpValve.java:680)\n\tat
org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:118)\n\tat
org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:407)\n\tat
org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1002)\n\tat
org.apache.coyote.AbstractProtocol$AbstractConnect
Re: Occasionally getting error in solr suggester component.
Hi,
>From Solr 4.7 onwards, the implementation of this Suggester is changed. The
old SpellChecker based search component is replaced with a new suggester
that utilizes Lucene suggester module. The latest Solr download is
preconfigured with this new suggester
I;m using Solr 4.10 and suggestion are based on query /suggest instead of
/spell.
So what I did is that in changed to false
Its not good that each time rebuild the index on commit , however, I
would like to build
the index on certain time period, say 1 hour.
The lookup data will be built only when requested by URL parameter
suggest.build=true
"http://localhost:8983/solr/ha/suggest?suggest.build=true";
So this will rebuild the index again and the changes will reflect in the
suggester.
There are certain pros and cons for this.
Issue is that the change will reflect only on certain time interval, here 1
hour. Advantage is that we can avoid the rebuilt index on every commit or
optimize.
Is this the right way ?? or any that I missed ???
Regards
dhanesh s.r
On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
> did you build the spellcheck index using spellcheck.build as described
> here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ?
>
> -Mike
>
>
> On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote:
>
>> Hi,
>> Thanks for the reply.
>> As you mentioned in the previous mail I changed buildOnCommit=false in
>> solrConfig.
>> After that change, suggestions are not working.
>> In Solr 4.7 introduced a new approach based on a dedicated
>> SuggestComponent
>> I'm using that component to build suggestions and lookup implementation is
>> "AnalyzingInfixLookupFactory"
>> Is there any work around ??
>>
>>
>>
>>
>> On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov <
>> msoko...@safaribooksonline.com> wrote:
>>
>> I think you are probably getting bitten by one of the issues addressed in
>>> LUCENE-5889
>>>
>>> I would recommend against using buildOnCommit=true - with a large index
>>> this can be a performance-killer. Instead, build the index yourself
>>> using
>>> the Solr spellchecker support (spellcheck.build=true)
>>>
>>> -Mike
>>>
>>>
>>> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
>>>
>>> Hi all,
>>>>
>>>> I am experiencing a problem in Solr SuggestComponent
>>>> Occasionally solr suggester component throws an error like
>>>>
>>>> Solr failed:
>>>> {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester
>>>> was
>>>> not built","trace":"java.lang.IllegalStateException: suggester was not
>>>> built\n\tat
>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>> lookup(AnalyzingInfixSuggester.java:368)\n\tat
>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>> lookup(AnalyzingInfixSuggester.java:342)\n\tat
>>>> org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
>>>> org.apache.solr.spelling.suggest.SolrSuggester.
>>>> getSuggestions(SolrSuggester.java:199)\n\tat
>>>> org.apache.solr.handler.component.SuggestComponent.
>>>> process(SuggestComponent.java:234)\n\tat
>>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(
>>>> SearchHandler.java:218)\n\tat
>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(
>>>> RequestHandlerBase.java:135)\n\tat
>>>> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
>>>> handleRequest(RequestHandlers.java:246)\n\tat
>>>> org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
>>>> org.apache.solr.servlet.SolrDispatchFilter.execute(
>>>> SolrDispatchFilter.java:777)\n\tat
>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>> SolrDispatchFilter.java:418)\n\tat
>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>> SolrDispatchFilter.java:207)\n\tat
>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>>>> ApplicationFilterChain.java:243)\n\tat
>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(
>>>> ApplicationFilterChain.java:210)\n\tat
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(
>>>> StandardWrapperValve.java:225)\n\tat
>>>> org.apache.catalina.core.
Re: Occasionally getting error in solr suggester component.
did you build the spellcheck index using spellcheck.build as described
here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ?
-Mike
On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote:
Hi,
Thanks for the reply.
As you mentioned in the previous mail I changed buildOnCommit=false in
solrConfig.
After that change, suggestions are not working.
In Solr 4.7 introduced a new approach based on a dedicated SuggestComponent
I'm using that component to build suggestions and lookup implementation is
"AnalyzingInfixLookupFactory"
Is there any work around ??
On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
I think you are probably getting bitten by one of the issues addressed in
LUCENE-5889
I would recommend against using buildOnCommit=true - with a large index
this can be a performance-killer. Instead, build the index yourself using
the Solr spellchecker support (spellcheck.build=true)
-Mike
On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
Hi all,
I am experiencing a problem in Solr SuggestComponent
Occasionally solr suggester component throws an error like
Solr failed:
{"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester was
not built","trace":"java.lang.IllegalStateException: suggester was not
built\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
lookup(AnalyzingInfixSuggester.java:368)\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
lookup(AnalyzingInfixSuggester.java:342)\n\tat
org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
org.apache.solr.spelling.suggest.SolrSuggester.
getSuggestions(SolrSuggester.java:199)\n\tat
org.apache.solr.handler.component.SuggestComponent.
process(SuggestComponent.java:234)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(
SearchHandler.java:218)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(
RequestHandlerBase.java:135)\n\tat
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
handleRequest(RequestHandlers.java:246)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(
SolrDispatchFilter.java:777)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(
SolrDispatchFilter.java:418)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(
SolrDispatchFilter.java:207)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
ApplicationFilterChain.java:243)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(
ApplicationFilterChain.java:210)\n\tat
org.apache.catalina.core.StandardWrapperValve.invoke(
StandardWrapperValve.java:225)\n\tat
org.apache.catalina.core.StandardContextValve.invoke(
StandardContextValve.java:123)\n\tat
org.apache.catalina.core.StandardHostValve.invoke(
StandardHostValve.java:168)\n\tat
org.apache.catalina.valves.ErrorReportValve.invoke(
ErrorReportValve.java:98)\n\tat
org.apache.catalina.valves.AccessLogValve.invoke(
AccessLogValve.java:927)\n\tat
org.apache.catalina.valves.RemoteIpValve.invoke(
RemoteIpValve.java:680)\n\tat
org.apache.catalina.core.StandardEngineValve.invoke(
StandardEngineValve.java:118)\n\tat
org.apache.catalina.connector.CoyoteAdapter.service(
CoyoteAdapter.java:407)\n\tat
org.apache.coyote.http11.AbstractHttp11Processor.process(
AbstractHttp11Processor.java:1002)\n\tat
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
process(AbstractProtocol.java:579)\n\tat
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
run(JIoEndpoint.java:312)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1145)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:745)\n","code":500}}
This is not freequently happening, but idexing and suggestor component
working togethere this error will occur.
In solr config
haSuggester
AnalyzingInfixLookupFactory
textSpell
DocumentDictionaryFactory
name
packageWeight
true
true
10
suggest
Can any one suggest where to look to figure out this error and why these
errors are occurring?
Thanks,
dhanesh s.r
--
Re: Occasionally getting error in solr suggester component.
Hi,
Thanks for the reply.
As you mentioned in the previous mail I changed buildOnCommit=false in
solrConfig.
After that change, suggestions are not working.
In Solr 4.7 introduced a new approach based on a dedicated SuggestComponent
I'm using that component to build suggestions and lookup implementation is
"AnalyzingInfixLookupFactory"
Is there any work around ??
On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
> I think you are probably getting bitten by one of the issues addressed in
> LUCENE-5889
>
> I would recommend against using buildOnCommit=true - with a large index
> this can be a performance-killer. Instead, build the index yourself using
> the Solr spellchecker support (spellcheck.build=true)
>
> -Mike
>
>
> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
>
>> Hi all,
>>
>> I am experiencing a problem in Solr SuggestComponent
>> Occasionally solr suggester component throws an error like
>>
>> Solr failed:
>> {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester was
>> not built","trace":"java.lang.IllegalStateException: suggester was not
>> built\n\tat
>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>> lookup(AnalyzingInfixSuggester.java:368)\n\tat
>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>> lookup(AnalyzingInfixSuggester.java:342)\n\tat
>> org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
>> org.apache.solr.spelling.suggest.SolrSuggester.
>> getSuggestions(SolrSuggester.java:199)\n\tat
>> org.apache.solr.handler.component.SuggestComponent.
>> process(SuggestComponent.java:234)\n\tat
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(
>> SearchHandler.java:218)\n\tat
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(
>> RequestHandlerBase.java:135)\n\tat
>> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
>> handleRequest(RequestHandlers.java:246)\n\tat
>> org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.execute(
>> SolrDispatchFilter.java:777)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:418)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:207)\n\tat
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:243)\n\tat
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:210)\n\tat
>> org.apache.catalina.core.StandardWrapperValve.invoke(
>> StandardWrapperValve.java:225)\n\tat
>> org.apache.catalina.core.StandardContextValve.invoke(
>> StandardContextValve.java:123)\n\tat
>> org.apache.catalina.core.StandardHostValve.invoke(
>> StandardHostValve.java:168)\n\tat
>> org.apache.catalina.valves.ErrorReportValve.invoke(
>> ErrorReportValve.java:98)\n\tat
>> org.apache.catalina.valves.AccessLogValve.invoke(
>> AccessLogValve.java:927)\n\tat
>> org.apache.catalina.valves.RemoteIpValve.invoke(
>> RemoteIpValve.java:680)\n\tat
>> org.apache.catalina.core.StandardEngineValve.invoke(
>> StandardEngineValve.java:118)\n\tat
>> org.apache.catalina.connector.CoyoteAdapter.service(
>> CoyoteAdapter.java:407)\n\tat
>> org.apache.coyote.http11.AbstractHttp11Processor.process(
>> AbstractHttp11Processor.java:1002)\n\tat
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
>> process(AbstractProtocol.java:579)\n\tat
>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
>> run(JIoEndpoint.java:312)\n\tat
>> java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1145)\n\tat
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:615)\n\tat
>> java.lang.Thread.run(Thread.java:745)\n","code":500}}
>>
>> This is not freequently happening, but idexing and suggestor component
>> working togethere this error will occur.
>>
>>
>>
>>
>> In solr config
>>
>>
>>
>>haSuggester
>>AnalyzingInfixLookupFactory
>>textSpell
>>DocumentDictionaryFactory
>>
>>name
>>packageWeight
>>true
>>
>>
>>
>>> startup="lazy">
>>
>>true
>>10
>>
>>
>>sugges
Re: Occasionally getting error in solr suggester component.
Related question -
I see mention of needing to rebuild the spellcheck/suggest dictionary after
solr core reload. I see spellcheckIndexDir in both the old wiki entry and
the solr reference guide
<https://cwiki.apache.org/confluence/display/solr/Spell+Checking>. If this
parameter is provided, it sounds like the index is stored on the filesystem
and need not be rebuilt each time the core is reloaded.
Is this a correct understanding?
On Tue, Jan 13, 2015 at 2:17 PM, Michael Sokolov <
msoko...@safaribooksonline.com> wrote:
> I think you are probably getting bitten by one of the issues addressed in
> LUCENE-5889
>
> I would recommend against using buildOnCommit=true - with a large index
> this can be a performance-killer. Instead, build the index yourself using
> the Solr spellchecker support (spellcheck.build=true)
>
> -Mike
>
>
> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
>
>> Hi all,
>>
>> I am experiencing a problem in Solr SuggestComponent
>> Occasionally solr suggester component throws an error like
>>
>> Solr failed:
>> {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester was
>> not built","trace":"java.lang.IllegalStateException: suggester was not
>> built\n\tat
>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>> lookup(AnalyzingInfixSuggester.java:368)\n\tat
>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>> lookup(AnalyzingInfixSuggester.java:342)\n\tat
>> org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
>> org.apache.solr.spelling.suggest.SolrSuggester.
>> getSuggestions(SolrSuggester.java:199)\n\tat
>> org.apache.solr.handler.component.SuggestComponent.
>> process(SuggestComponent.java:234)\n\tat
>> org.apache.solr.handler.component.SearchHandler.handleRequestBody(
>> SearchHandler.java:218)\n\tat
>> org.apache.solr.handler.RequestHandlerBase.handleRequest(
>> RequestHandlerBase.java:135)\n\tat
>> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
>> handleRequest(RequestHandlers.java:246)\n\tat
>> org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.execute(
>> SolrDispatchFilter.java:777)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:418)\n\tat
>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:207)\n\tat
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>> ApplicationFilterChain.java:243)\n\tat
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(
>> ApplicationFilterChain.java:210)\n\tat
>> org.apache.catalina.core.StandardWrapperValve.invoke(
>> StandardWrapperValve.java:225)\n\tat
>> org.apache.catalina.core.StandardContextValve.invoke(
>> StandardContextValve.java:123)\n\tat
>> org.apache.catalina.core.StandardHostValve.invoke(
>> StandardHostValve.java:168)\n\tat
>> org.apache.catalina.valves.ErrorReportValve.invoke(
>> ErrorReportValve.java:98)\n\tat
>> org.apache.catalina.valves.AccessLogValve.invoke(
>> AccessLogValve.java:927)\n\tat
>> org.apache.catalina.valves.RemoteIpValve.invoke(
>> RemoteIpValve.java:680)\n\tat
>> org.apache.catalina.core.StandardEngineValve.invoke(
>> StandardEngineValve.java:118)\n\tat
>> org.apache.catalina.connector.CoyoteAdapter.service(
>> CoyoteAdapter.java:407)\n\tat
>> org.apache.coyote.http11.AbstractHttp11Processor.process(
>> AbstractHttp11Processor.java:1002)\n\tat
>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
>> process(AbstractProtocol.java:579)\n\tat
>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
>> run(JIoEndpoint.java:312)\n\tat
>> java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1145)\n\tat
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:615)\n\tat
>> java.lang.Thread.run(Thread.java:745)\n","code":500}}
>>
>> This is not freequently happening, but idexing and suggestor component
>> working togethere this error will occur.
>>
>>
>>
>>
>> In solr config
>>
>>
>>
>>haSuggester
>>AnalyzingInfixLookupFactory
>>textSpell
>>DocumentDictionaryFactory
>>
>>name
>>packageWeight
>>true
>>
>>
>>
>>> startup="lazy">
>>
>>true
>>10
>>
>>
>>suggest
>>
>>
>>
>> Can any one suggest where to look to figure out this error and why these
>> errors are occurring?
>>
>>
>>
>> Thanks,
>> dhanesh s.r
>>
>>
>>
>>
>> --
>>
>>
>
Re: Occasionally getting error in solr suggester component.
I think you are probably getting bitten by one of the issues addressed
in LUCENE-5889
I would recommend against using buildOnCommit=true - with a large index
this can be a performance-killer. Instead, build the index yourself
using the Solr spellchecker support (spellcheck.build=true)
-Mike
On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
Hi all,
I am experiencing a problem in Solr SuggestComponent
Occasionally solr suggester component throws an error like
Solr failed:
{"responseHeader":{"status":500,"QTime":1},"error":{"msg":"suggester was
not built","trace":"java.lang.IllegalStateException: suggester was not
built\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:368)\n\tat
org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:342)\n\tat
org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n\tat
org.apache.solr.spelling.suggest.SolrSuggester.getSuggestions(SolrSuggester.java:199)\n\tat
org.apache.solr.handler.component.SuggestComponent.process(SuggestComponent.java:234)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:246)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)\n\tat
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)\n\tat
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)\n\tat
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)\n\tat
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)\n\tat
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)\n\tat
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)\n\tat
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:680)\n\tat
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)\n\tat
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)\n\tat
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1002)\n\tat
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)\n\tat
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:745)\n","code":500}}
This is not freequently happening, but idexing and suggestor component
working togethere this error will occur.
In solr config
haSuggester
AnalyzingInfixLookupFactory
textSpell
DocumentDictionaryFactory
name
packageWeight
true
true
10
suggest
Can any one suggest where to look to figure out this error and why these
errors are occurring?
Thanks,
dhanesh s.r
--
