subject:"Spelling suggestions."

RE: Can't get spelling suggestions to work properly

2017-01-17 Thread Dyer, James

Jimi,

Generally speaking, spellcheck does not work well against fields with stemming, 
or other "heavy" analysis.  I would  to a field that is tokenized 
on whitespace with little else, and use that field for spellcheck.

By default, the spellchecker does not suggest for words in the index.  So if 
the user misspells a word but the misspelling is actually some other word that 
is indexed, it will never suggest.  You can orverride this behavior by 
specifying  "spellcheck.alternativeTermCount" with a value >0.  This is how 
many suggestions it should give for words that indeed exist in the index.  This 
can be the same value as "spellcheck.count", but you may wish to set it to a 
lower value.

I do not recommend using "spellcheck.onlyMorePopular".  It is similar to 
"spellcheck.alternativeTermCount", but in my opinion, the later gives a better 
experience.

You might also wish to set "spellcheck.maxResultsForSuggest".  If you set this, 
then the spellchecker will not suggest anything if more results are returned 
than the value you specify.  This is helpful in providing "did you mean"-style 
suggestions for queries that return few results.

If you would like to ensure the suggestions combine nicely into a re-written 
query that returns results, then specify both "spellcheck.collate=true" and 
"spellcheck.maxCollationTries" to a value >0 (possibly 5-10).  This will cause 
it to internally check the re-written queries (aka. Collations) and report back 
on how many results you get for each.  If you are using "q.op=OR" or a low 
value for "mm", then you will likely want to override this with something like 
"spellcheck.collateParam.mm=0".  Otherwise every combination will get reported 
as returning results.

I hope this and other comments you've gotten helps demystify spellcheck 
configuration.  I do agree it is fairly complicated and frustrating to get it 
just right.

James Dyer
Ingram Content Group

-Original Message-
From: jimi.hulleg...@svensktnaringsliv.se 
[mailto:jimi.hulleg...@svensktnaringsliv.se] 
Sent: Friday, January 13, 2017 5:16 AM
To: solr-user@lucene.apache.org
Subject: RE: Can't get spelling suggestions to work properly

I just noticed why setting maxResultsForSuggest to a high value was not a good 
thing. Because now it show spelling suggestions even on correctly spelled words.

I think, what I would need is the logic of SuggestMode. 
SUGGEST_WHEN_NOT_IN_INDEX, but with a configurable limit instead of it being 
hard coded to 0. Ie just as maxQueryFrequency works.

/Jimi

-Original Message-
From: jimi.hulleg...@svensktnaringsliv.se 
[mailto:jimi.hulleg...@svensktnaringsliv.se] 
Sent: Friday, January 13, 2017 5:56 PM
To: solr-user@lucene.apache.org
Subject: RE: Can't get spelling suggestions to work properly

Hi Alessandro,

Thanks for your explanation. It helped a lot. Although setting 
"spellcheck.maxResultsForSuggest" to a value higher than zero was not enough. I 
also had to set "spellcheck.alternativeTermCount". With that done, I now get 
suggestions when searching for 'mycet' (a misspelling of the Swedish word 
'mycket', that didn't return suggestions before).

Although, I'm still not able to fully understand how to configure this 
properly. Because with this change there now are other misspelled searches that 
now longer gives suggestions. The problem here is stemming, I suspect. Because 
the main search fields use stemming, so that in some cases one can get lots of 
results for spellings that doesn't exist in the index at all (or, at least not 
in the spelling-field). How can I configure this component so that those 
suggestions are still included? Do I need to set maxResultsForSuggest to a 
really high number? Like Integer.MAX_VALUE? I feel that such a setting would 
defeat the purpose of that parameter, in a way. But I'm not sure how else to 
solve this.

Also, there is one other things I wonder about the spelling suggestions, that 
you might have the answer to. Is there a way to make the logic case 
insensitive, but the presentation case sensitive? For example, a search for 
'georg washington' now would return 'george washington' as a suggestion, but ' 
Georg Washington' would be even better.

Regards
/Jimi


-Original Message-
From: alessandro.benedetti [mailto:abenede...@apache.org] 
Sent: Thursday, January 12, 2017 5:14 PM
To: solr-user@lucene.apache.org
Subject: Re: Can't get spelling suggestions to work properly

Hi Jimi,
taking a look to the *maxQueryFrequency*  param :

Your understanding is correct.

1) we don't provide misspelled suggestions if we set the param to 1, and we 
have a minimum of 1 doc freq for the term .

2) we don't provide misspelled suggestions if the doc frequency of the term is 
greater than the max limit set.

Let us explore the code :

if (suggestMode==SuggestMode.SUGGEST_WHEN_NOT_IN_

RE: Can't get spelling suggestions to work properly

2017-01-13 Thread jimi.hullegard

I just noticed why setting maxResultsForSuggest to a high value was not a good 
thing. Because now it show spelling suggestions even on correctly spelled words.

I think, what I would need is the logic of SuggestMode. 
SUGGEST_WHEN_NOT_IN_INDEX, but with a configurable limit instead of it being 
hard coded to 0. Ie just as maxQueryFrequency works.

/Jimi

-Original Message-
From: jimi.hulleg...@svensktnaringsliv.se 
[mailto:jimi.hulleg...@svensktnaringsliv.se] 
Sent: Friday, January 13, 2017 5:56 PM
To: solr-user@lucene.apache.org
Subject: RE: Can't get spelling suggestions to work properly

Hi Alessandro,

Thanks for your explanation. It helped a lot. Although setting 
"spellcheck.maxResultsForSuggest" to a value higher than zero was not enough. I 
also had to set "spellcheck.alternativeTermCount". With that done, I now get 
suggestions when searching for 'mycet' (a misspelling of the Swedish word 
'mycket', that didn't return suggestions before).

Although, I'm still not able to fully understand how to configure this 
properly. Because with this change there now are other misspelled searches that 
now longer gives suggestions. The problem here is stemming, I suspect. Because 
the main search fields use stemming, so that in some cases one can get lots of 
results for spellings that doesn't exist in the index at all (or, at least not 
in the spelling-field). How can I configure this component so that those 
suggestions are still included? Do I need to set maxResultsForSuggest to a 
really high number? Like Integer.MAX_VALUE? I feel that such a setting would 
defeat the purpose of that parameter, in a way. But I'm not sure how else to 
solve this.

Also, there is one other things I wonder about the spelling suggestions, that 
you might have the answer to. Is there a way to make the logic case 
insensitive, but the presentation case sensitive? For example, a search for 
'georg washington' now would return 'george washington' as a suggestion, but ' 
Georg Washington' would be even better.

Regards
/Jimi


-Original Message-
From: alessandro.benedetti [mailto:abenede...@apache.org] 
Sent: Thursday, January 12, 2017 5:14 PM
To: solr-user@lucene.apache.org
Subject: Re: Can't get spelling suggestions to work properly

Hi Jimi,
taking a look to the *maxQueryFrequency*  param :

Your understanding is correct.

1) we don't provide misspelled suggestions if we set the param to 1, and we 
have a minimum of 1 doc freq for the term .

2) we don't provide misspelled suggestions if the doc frequency of the term is 
greater than the max limit set.

Let us explore the code :

if (suggestMode==SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX && docfreq > 0) {
  return new SuggestWord[0];
}
/// If we are working in "Not in Index Mode" , with a document frequency >0 we 
get no misspelled corrections.
/

int maxDoc = ir.maxDoc();

if (maxQueryFrequency >= 1f && docfreq > maxQueryFrequency) {
  return new SuggestWord[0];
} else if (docfreq > (int) Math.ceil(maxQueryFrequency * (float)maxDoc)) {
  return new SuggestWord[0];
}
// then the MaxQueryFrequency as you correctly stated enters the game

...

Let's explore how you can end up in the first scenario :

if (maxResultsForSuggest == null || hits <= maxResultsForSuggest) {
  SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;
  if (onlyMorePopular) {
suggestMode = SuggestMode.SUGGEST_MORE_POPULAR;
  } else if (alternativeTermCount > 0) {
suggestMode = SuggestMode.SUGGEST_ALWAYS;
  }

You did not set maxResultsForSuggest ( and not onlyMorePopular or alternative 
term count) so you ended up in :
SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;

>From Solr javaDoc :

If left unspecified, the default behavior will prevail.  That is, 
"correctlySpelled" will be false and suggestions
   * will be returned only if one or more of the query terms are absent from 
the dictionary and/or index.  If set to zero,
   * the "correctlySpelled" flag will be false only if the response returns 
zero hits.  If set to a value greater than zero, 
   * suggestions will be returned even if hits are returned (up to the 
specified number).  This number also will serve as
   * the threshold in determining the value of "correctlySpelled". 
Specifying a value greater than zero is useful 
   * for creating "did-you-mean" suggestions for queries that return a low 
number of hits.
   * 
   */
  public static final String SPELLCHECK_MAX_RESULTS_FOR_SUGGEST = 
SPELLCHECK_PREFIX + "maxResultsForSuggest";

You probably want to bypass the other parameters and just set the proper 
maxResultsForSuggest param for your spellchecker Cheers



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Can-t-get-spelling-suggestions-to-work-properly-tp4310079p4313685.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Can't get spelling suggestions to work properly

2017-01-13 Thread jimi.hullegard

Hi Alessandro,

Thanks for your explanation. It helped a lot. Although setting 
"spellcheck.maxResultsForSuggest" to a value higher than zero was not enough. I 
also had to set "spellcheck.alternativeTermCount". With that done, I now get 
suggestions when searching for 'mycet' (a misspelling of the Swedish word 
'mycket', that didn't return suggestions before).

Although, I'm still not able to fully understand how to configure this 
properly. Because with this change there now are other misspelled searches that 
now longer gives suggestions. The problem here is stemming, I suspect. Because 
the main search fields use stemming, so that in some cases one can get lots of 
results for spellings that doesn't exist in the index at all (or, at least not 
in the spelling-field). How can I configure this component so that those 
suggestions are still included? Do I need to set maxResultsForSuggest to a 
really high number? Like Integer.MAX_VALUE? I feel that such a setting would 
defeat the purpose of that parameter, in a way. But I'm not sure how else to 
solve this.

Also, there is one other things I wonder about the spelling suggestions, that 
you might have the answer to. Is there a way to make the logic case 
insensitive, but the presentation case sensitive? For example, a search for 
'georg washington' now would return 'george washington' as a suggestion, but ' 
Georg Washington' would be even better.

Regards
/Jimi


-Original Message-
From: alessandro.benedetti [mailto:abenede...@apache.org] 
Sent: Thursday, January 12, 2017 5:14 PM
To: solr-user@lucene.apache.org
Subject: Re: Can't get spelling suggestions to work properly

Hi Jimi,
taking a look to the *maxQueryFrequency*  param :

Your understanding is correct.

1) we don't provide misspelled suggestions if we set the param to 1, and we 
have a minimum of 1 doc freq for the term .

2) we don't provide misspelled suggestions if the doc frequency of the term is 
greater than the max limit set.

Let us explore the code :

if (suggestMode==SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX && docfreq > 0) {
  return new SuggestWord[0];
}
/// If we are working in "Not in Index Mode" , with a document frequency >0 we 
get no misspelled corrections.
/

int maxDoc = ir.maxDoc();

if (maxQueryFrequency >= 1f && docfreq > maxQueryFrequency) {
  return new SuggestWord[0];
} else if (docfreq > (int) Math.ceil(maxQueryFrequency * (float)maxDoc)) {
  return new SuggestWord[0];
}
// then the MaxQueryFrequency as you correctly stated enters the game

...

Let's explore how you can end up in the first scenario :

if (maxResultsForSuggest == null || hits <= maxResultsForSuggest) {
  SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;
  if (onlyMorePopular) {
suggestMode = SuggestMode.SUGGEST_MORE_POPULAR;
  } else if (alternativeTermCount > 0) {
suggestMode = SuggestMode.SUGGEST_ALWAYS;
  }

You did not set maxResultsForSuggest ( and not onlyMorePopular or alternative 
term count) so you ended up in :
SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;

>From Solr javaDoc :

If left unspecified, the default behavior will prevail.  That is, 
"correctlySpelled" will be false and suggestions
   * will be returned only if one or more of the query terms are absent from 
the dictionary and/or index.  If set to zero,
   * the "correctlySpelled" flag will be false only if the response returns 
zero hits.  If set to a value greater than zero, 
   * suggestions will be returned even if hits are returned (up to the 
specified number).  This number also will serve as
   * the threshold in determining the value of "correctlySpelled". 
Specifying a value greater than zero is useful 
   * for creating "did-you-mean" suggestions for queries that return a low 
number of hits.
   * 
   */
  public static final String SPELLCHECK_MAX_RESULTS_FOR_SUGGEST = 
SPELLCHECK_PREFIX + "maxResultsForSuggest";

You probably want to bypass the other parameters and just set the proper 
maxResultsForSuggest param for your spellchecker Cheers



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Can-t-get-spelling-suggestions-to-work-properly-tp4310079p4313685.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Can't get spelling suggestions to work properly

2017-01-12 Thread alessandro.benedetti

Hi Jimi,
taking a look to the *maxQueryFrequency*  param :

Your understanding is correct.

1) we don't provide misspelled suggestions if we set the param to 1, and we
have a minimum of 1 doc freq for the term .

2) we don't provide misspelled suggestions if the doc frequency of the term
is greater than the max limit set.

Let us explore the code :

if (suggestMode==SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX && docfreq > 0) {
  return new SuggestWord[0];
}
/// If we are working in "Not in Index Mode" , with a document frequency >0
we get no misspelled corrections.
/

int maxDoc = ir.maxDoc();

if (maxQueryFrequency >= 1f && docfreq > maxQueryFrequency) {
  return new SuggestWord[0];
} else if (docfreq > (int) Math.ceil(maxQueryFrequency * (float)maxDoc))
{
  return new SuggestWord[0];
}
// then the MaxQueryFrequency as you correctly stated enters the game

...

Let's explore how you can end up in the first scenario :

if (maxResultsForSuggest == null || hits <= maxResultsForSuggest) {
  SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;
  if (onlyMorePopular) {
suggestMode = SuggestMode.SUGGEST_MORE_POPULAR;
  } else if (alternativeTermCount > 0) {
suggestMode = SuggestMode.SUGGEST_ALWAYS;
  }

You did not set maxResultsForSuggest ( and not onlyMorePopular or
alternative term count) so you ended up in :
SuggestMode suggestMode = SuggestMode.SUGGEST_WHEN_NOT_IN_INDEX;

>From Solr javaDoc :

If left unspecified, the default behavior will prevail.  That is,
"correctlySpelled" will be false and suggestions
   * will be returned only if one or more of the query terms are absent from
the dictionary and/or index.  If set to zero,
   * the "correctlySpelled" flag will be false only if the response returns
zero hits.  If set to a value greater than zero, 
   * suggestions will be returned even if hits are returned (up to the
specified number).  This number also will serve as
   * the threshold in determining the value of "correctlySpelled". 
Specifying a value greater than zero is useful 
   * for creating "did-you-mean" suggestions for queries that return a low
number of hits.
   * 
   */
  public static final String SPELLCHECK_MAX_RESULTS_FOR_SUGGEST =
SPELLCHECK_PREFIX + "maxResultsForSuggest";

You probably want to bypass the other parameters and just set the proper
maxResultsForSuggest param for your spellchecker
Cheers



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Can-t-get-spelling-suggestions-to-work-properly-tp4310079p4313685.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Can't get spelling suggestions to work properly

2017-01-12 Thread Matt Pearce


Hi Jimi,

It looks like the suggest mode defaults to only returning results when 
the query term is not in your index.


I think setting spellcheck.onlyMorePopular=true, or a value for 
spellcheck.alternativeTermCount should change the suggest mode to one 
that doesn't enforce that restriction. Apologies if you've already tried 
those, but it doesn't look like they're set in your query params below.


I know from recent experience that setting up spell checkers (and 
suggesters) can be an irritating process, and often a result of lots of 
trial and error testing!


All the best,

Matt


On 10/01/17 15:41, jimi.hulleg...@svensktnaringsliv.se wrote:

No one has any input on my post below about the spelling suggestions? I just find it a 
bit frustrating not being able to understand this feature better, and why it doesn't give 
the expected results. A built in "explain" feature really would have helped.

/Jimi

-Original Message-
From: jimi.hulleg...@svensktnaringsliv.se 
[mailto:jimi.hulleg...@svensktnaringsliv.se]
Sent: Friday, December 16, 2016 9:58 PM
To: solr-user@lucene.apache.org
Subject: Can't get spelling suggestions to work properly

Hi,

I'm trying to add the spelling suggestion feature to our search, but I'm having 
problems getting suggestions on some misspellings.

For example, the Swedish word 'mycket' exists in ~14.000 of a total of ~40.000 
documents in our index.

A search for the incorrect spelling 'myket' (a missing 'c') gives several 
spelling suggestions, and the top one is 'mycket'. This is the wanted/expected 
behaivor.

But a search for the incorrect spelling 'mycet' (a missing 'k') gives no 
spelling suggestions.

The only difference between these two searches is that the one that results in 
spelling suggestions had zero results, while the other one had two (2) results. 
These two documents contain this incorrect spelling ('mycet'). Can this be the 
cause of no spelling suggestions? But I have set 'maxQueryFrequency' to 0.001, 
and with 40.000 documents in the index that should mean that the word can exist 
in up to 40 documents, and since 2 is less than 40 I argue that that this word 
would be considered a spelling misstake. But for some reason the solr 
spellchecker considers 'myket' as an incorrect spelling, while 'mycet' 
incorrectly is considered as a correct spelling.

Also, I tried with spellcheck.accuracy=0 just to rule out that I have a too 
high accuracy setting, but that didn't help.

Can someone see what I'm doing wrong, or give some tips on configuration 
changes and/or how I can troubleshoot this? For example, is there any way to 
debug the spellchecker function?


Here are the searches:

Search for 'myket':

http://localhost:8080/solr/s2/select/?q=myket=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Spellcheck output for 'myket':


 
  
   16
   0
   5
   0
   
  
  
  
   mycket
  
   14039
  
  

[...]
  
 
  
  false
  
   mycket
   14005

RE: Can't get spelling suggestions to work properly

2017-01-10 Thread jimi.hullegard

No one has any input on my post below about the spelling suggestions? I just 
find it a bit frustrating not being able to understand this feature better, and 
why it doesn't give the expected results. A built in "explain" feature really 
would have helped.

/Jimi

-Original Message-
From: jimi.hulleg...@svensktnaringsliv.se 
[mailto:jimi.hulleg...@svensktnaringsliv.se] 
Sent: Friday, December 16, 2016 9:58 PM
To: solr-user@lucene.apache.org
Subject: Can't get spelling suggestions to work properly

Hi,

I'm trying to add the spelling suggestion feature to our search, but I'm having 
problems getting suggestions on some misspellings.

For example, the Swedish word 'mycket' exists in ~14.000 of a total of ~40.000 
documents in our index.

A search for the incorrect spelling 'myket' (a missing 'c') gives several 
spelling suggestions, and the top one is 'mycket'. This is the wanted/expected 
behaivor.

But a search for the incorrect spelling 'mycet' (a missing 'k') gives no 
spelling suggestions.

The only difference between these two searches is that the one that results in 
spelling suggestions had zero results, while the other one had two (2) results. 
These two documents contain this incorrect spelling ('mycet'). Can this be the 
cause of no spelling suggestions? But I have set 'maxQueryFrequency' to 0.001, 
and with 40.000 documents in the index that should mean that the word can exist 
in up to 40 documents, and since 2 is less than 40 I argue that that this word 
would be considered a spelling misstake. But for some reason the solr 
spellchecker considers 'myket' as an incorrect spelling, while 'mycet' 
incorrectly is considered as a correct spelling.

Also, I tried with spellcheck.accuracy=0 just to rule out that I have a too 
high accuracy setting, but that didn't help.

Can someone see what I'm doing wrong, or give some tips on configuration 
changes and/or how I can troubleshoot this? For example, is there any way to 
debug the spellchecker function?


Here are the searches:

Search for 'myket':

http://localhost:8080/solr/s2/select/?q=myket=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Spellcheck output for 'myket':


 
  

   16

   0

   5

   0

   



 mycket

 14039



[...]

   
  
  false
  

   mycket

   14005

   

mycket

   
  
  [...]
  
 



Spellcheck output for 'mycet':

http://localhost:8080/solr/s2/select/?q=mycet=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Search for 'myc

Can't get spelling suggestions to work properly

2016-12-16 Thread jimi.hullegard

Hi,

I'm trying to add the spelling suggestion feature to our search, but I'm having 
problems getting suggestions on some misspellings.

For example, the Swedish word 'mycket' exists in ~14.000 of a total of ~40.000 
documents in our index.

A search for the incorrect spelling 'myket' (a missing 'c') gives several 
spelling suggestions, and the top one is 'mycket'. This is the wanted/expected 
behaivor.

But a search for the incorrect spelling 'mycet' (a missing 'k') gives no 
spelling suggestions.

The only difference between these two searches is that the one that results in 
spelling suggestions had zero results, while the other one had two (2) results. 
These two documents contain this incorrect spelling ('mycet'). Can this be the 
cause of no spelling suggestions? But I have set 'maxQueryFrequency' to 0.001, 
and with 40.000 documents in the index that should mean that the word can exist 
in up to 40 documents, and since 2 is less than 40 I argue that that this word 
would be considered a spelling misstake. But for some reason the solr 
spellchecker considers 'myket' as an incorrect spelling, while 'mycet' 
incorrectly is considered as a correct spelling.

Also, I tried with spellcheck.accuracy=0 just to rule out that I have a too 
high accuracy setting, but that didn't help.

Can someone see what I'm doing wrong, or give some tips on configuration 
changes and/or how I can troubleshoot this? For example, is there any way to 
debug the spellchecker function?


Here are the searches:

Search for 'myket':

http://localhost:8080/solr/s2/select/?q=myket=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Spellcheck output for 'myket':


 
  

   16

   0

   5

   0

   



 mycket

 14039



[...]

   
  
  false
  

   mycket

   14005

   

mycket

   
  
  [...]
  
 



Spellcheck output for 'mycet':

http://localhost:8080/solr/s2/select/?q=mycet=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Search for 'mycet':

http://localhost:8080/solr/s2/select/?q=mycet=100=score+desc=*%2Cscore%2C%5Bexplain+style%3Dtext%5D=edismax=title%5E2=swedishText1%5E1=true=0=200=%2Bactivatedate%3A%5B*+TO+NOW%5D+%2Bexpiredate%3A%5BNOW+TO+*%5D+%2B%28state%3Apublished+OR+state%3Adraft-published+OR+state%3Asubmitted-published+OR+state%3Aapproved-published%29=xml=true

Spellcheck output:


 
  true
 



Below is the relevant configuration

Spelling suggestions--any ideas?

2014-04-17 Thread Ed Smiley

Correctly spelled words are returning as not spelled correctly, with the 
original, correctly spelled word with a single oddball character appended as 
multiple suggestions...
--

Ed Smiley, Senior Software Architect, eBooks
ProQuest | 161 E Evelyn Ave|
Mountain View, CA 94041 | USA |
+1 650 475 8700 extension 3772
ed.smi...@proquest.com
www.proquest.comhttp://www.proquest.com/ | 
www.ebrary.comhttp://www.ebrary.com/ | www.eblib.comhttp://www.eblib.com/
ebrary and EBL, ProQuest businesses.

Re: Spelling suggestions.

2013-08-11 Thread tamanjit.bin...@yahoo.co.in

I think the issue is that you are trying to use WordBreakSolrSpellChecker
(which was introduced in Solr 4.x version) in your Solr App of 3.5 version.
You need to correct that.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-suggestions-tp4083519p4083816.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling suggestions.

2013-08-10 Thread Kamaljeet Kaur


Actually I don't have much knowledge about the files and configuring
something with solr. Using apache-solr 3.5.0 and requesting the
following URL I got the result as shown in the attachment:

http://localhost:8983/solr/spell?q=delll
ultrasharspellcheck=truespellcheck.collate=truespellcheck.build=true

On editing solrconfig.xml, as given in the above link, restarting solr
given the error: http://tny.cz/cccf5c8f

Running http://localhost:8983/solr/ gives the error: http://tny.cz/0c533931

Here is my solrconfig.xml: http://tny.cz/4855ed9f
Where is the problem?




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-suggestions-tp4083519p4083715.html
Sent from the Solr - User mailing list archive at Nabble.com.

Spelling suggestions.

2013-08-09 Thread Kamaljeet Kaur

Hello,

I have just configured apache-solr with my django project. And its working
fine with a very simple and basic searching. I want to add spelling
suggestions, if user misspell any word in the string entered.

In this particular mailing-list, I searched for it. Many have give the link: 
http://wiki.apache.org/solr/SpellCheckComponent#head-78f5afcf43df544832809abc68dd36b98152670c

But I am using the version 3.5.0, Its for the version 1.3  Should i follow
this tutorial or they are available for solr version 3.5.0 ? 

Thanks
Kamaljeet Kaur



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-suggestions-tp4083519.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling suggestions.

2013-08-09 Thread Jason Hellman

The majority of the behavior outlined in that wiki page should work quite 
sufficiently for 3.5.0.  Note that there are only a few items that are marked 
Solr4.0 only (DirectSolrSpellChecker and WordBreakSolrSpellChecker, for 
example).

On Aug 9, 2013, at 6:26 AM, Kamaljeet Kaur kamal.kaur...@gmail.com wrote:

 Hello,
 
 I have just configured apache-solr with my django project. And its working
 fine with a very simple and basic searching. I want to add spelling
 suggestions, if user misspell any word in the string entered.
 
 In this particular mailing-list, I searched for it. Many have give the link: 
 http://wiki.apache.org/solr/SpellCheckComponent#head-78f5afcf43df544832809abc68dd36b98152670c
 
 But I am using the version 3.5.0, Its for the version 1.3  Should i follow
 this tutorial or they are available for solr version 3.5.0 ? 
 
 Thanks
 Kamaljeet Kaur
 
 
 
 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Spelling-suggestions-tp4083519.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Dyer, James

Andy,

I opened this ticket so that someone can eventaully invistigate: 
https://issues.apache.org/jira/browse/SOLR-4874

Just an instanity check, I see I had misspelled maxCollations as 
maxCollation in my prior response.  When you tested with this set the same as 
maxCollationTries, did you correct my spelling?  The thought is that by 
requiring it to return this many collations back, you are guaranteed to make it 
try the maximum time every time,giving yourself a cleaner test.  I am trying to 
isolate here if spellcheck is not running the queries properly or if the 
queries just naturally take that long to run over and over again.

James Dyer
Ingram Content Group
(615) 213-4311


-Original Message-
From: Andy Lester [mailto:a...@petdance.com] 
Sent: Tuesday, May 28, 2013 4:22 PM
To: solr-user@lucene.apache.org
Subject: Re: Why do FQs make my spelling suggestions so slow?

Thanks for looking at this.

 What are the QTimes for the 0fq,1fq,2fq,4fq  4fq cases with spellcheck 
 entirely turned off?  Is it about (or a little more than) half the total when 
 maxCollationTries=1 ?

With spellcheck off I get 8ms for 4fq query.


  Also, with the varying # of fq's, how many collation tries does it take to 
 get 10 collations?

I don't know.  How can I tell?


 Possibly, a better way to test this is to set maxCollations = 
 maxCollationTries.  The reason is that it quits trying once it finds 
 maxCollations, so if with 0fq's, lots of combinations can generate hits and 
 it doesn't need to try very many to get to 10.  But with more fq's, fewer 
 collations will pan out so now it is trying more up to 100 before (if ever) 
 it gets to 10.

It does just fine doing 100 collations so long as there are no FQs.  It seems 
to me that the FQs are taking an inordinate amount of extra time.  100 
collations in (roughly) the same amount of time as a single collation, so long 
as there are no FQs.  Why are the FQs such a drag on the collation process?


 (I'm assuming you have all non-search components like faceting turned off).

Yes, definitely.


  So say with 2fq's it takes 10ms for the query to complete with spellcheck 
 off, and 20ms with maxCollation = maxCollationTries = 1, then it will take 
 about 110ms with maxCollation = maxCollationTries = 10.

I can do maxCollation = maxCollationTries = 100 and it comes back in 14ms, so 
long as I have FQs off.  Add a single FQ and it becomes 13499ms.

I can do maxCollation = maxCollationTries = 1000 and it comes back in 45ms, so 
long as I have FQs off.  Add a single FQ and it becomes 62038ms.


 But I think you're just setting maxCollationTries too high.  You're asking it 
 to do too much work in trying teens of combinations.

The results I get back with 100 tries are about twice as many as I get with 10 
tries.  That's a big difference to the user where it's trying to figure 
misspelled phrases.

Andy

--
Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance

Re: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Andy Lester


On May 29, 2013, at 9:46 AM, Dyer, James james.d...@ingramcontent.com wrote:

 Just an instanity check, I see I had misspelled maxCollations as 
 maxCollation in my prior response.  When you tested with this set the same 
 as maxCollationTries, did you correct my spelling?

Yes, definitely.

Thanks for the ticket.  I am looking at the effects of turning on 
spellcheck.onlyMorePopular to true, which reduces the number of collations it 
seems to do, but doesn't affect the underlying question of is the spellchecker 
doing FQs properly?

Thanks,
Andy

--
Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance

Re: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Nicholas Fellows

I also have problems getting the solrspellchecker to utilise existing FQ
params correctly.
we have some fairly monster queries

eg : http://pastebin.com/4XzGpfeC

I cannot seem to get our FQ parameters to be honored when generating
results.
In essence i am getting collations that yield no results when the filter
query is applied.

We have items that are by default not shown when out of stock or
forthcoming. the user
can select whether to show these or not.

Is there something wrong with my query or perhaps my use case is not
supported?

Im using nested query and local params etc

Would very much appreciate some assistance on this one as 2days worth of
hacking, and pestering
people on IRC have not yet yeilded a solution for me. Im not even sure what
i am trying
is even possible! Some sort of clarification on this would really help!

Cheers

Nick...




On 29 May 2013 15:57, Andy Lester a...@petdance.com wrote:


 On May 29, 2013, at 9:46 AM, Dyer, James james.d...@ingramcontent.com
 wrote:

  Just an instanity check, I see I had misspelled maxCollations as
 maxCollation in my prior response.  When you tested with this set the
 same as maxCollationTries, did you correct my spelling?

 Yes, definitely.

 Thanks for the ticket.  I am looking at the effects of turning on
 spellcheck.onlyMorePopular to true, which reduces the number of collations
 it seems to do, but doesn't affect the underlying question of is the
 spellchecker doing FQs properly?

 Thanks,
 Andy

 --
 Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance




-- 
Nick Fellows
DJdownload.com
---
10 Greenland Street
London
NW10ND
United Kingdom
---
n...@djdownload.com (E)

---
www.djdownload.com

RE: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Dyer, James

Instead of maxCollationTries=0, use a value greater than zero.  Zero means 
not to check if the collation will return hits.  1 means to test 1 possible 
combination against the index and return it only if it returns hits.  2 tries 
up to 2 possibilities, etc.  As you have spellcheck.maxCollations=8, you'll 
probably want maxCollationTries at least that large.  Maybe 10-20 would be 
better.  Make it as low as possible to get generally good results, or as high 
as possible before the performance on a query with many misspelled words gets 
too bad.

Also, use a spellcheck.count greater than 2.  This is as many corrections per 
misspelled term you want it to consider.  If using DirectSolrSpellChecker, you 
can have it set low, 5-10 might be good.  If using IndexBased- or FileBased 
spell checkers, use at least 10.

Also, do not use onlyMorePopular unless you indeed want every term in the 
user's query to be replaced with higher-frequency terms (even correctly-spelled 
terms get replaced).  If you want it to suggest even for words that are in the 
dictionary, try spellcheck.alternativeTermCount instead.  Try setting it to 
about half of spellcheck.count (but at least 10 if using IndexBased- or 
FileBased spell checkers).

James Dyer
Ingram Content Group
(615) 213-4311


-Original Message-
From: Nicholas Fellows [mailto:n...@djdownload.com] 
Sent: Wednesday, May 29, 2013 11:06 AM
To: solr-user@lucene.apache.org
Subject: Re: Why do FQs make my spelling suggestions so slow?

I also have problems getting the solrspellchecker to utilise existing FQ
params correctly.
we have some fairly monster queries

eg : http://pastebin.com/4XzGpfeC

I cannot seem to get our FQ parameters to be honored when generating
results.
In essence i am getting collations that yield no results when the filter
query is applied.

We have items that are by default not shown when out of stock or
forthcoming. the user
can select whether to show these or not.

Is there something wrong with my query or perhaps my use case is not
supported?

Im using nested query and local params etc

Would very much appreciate some assistance on this one as 2days worth of
hacking, and pestering
people on IRC have not yet yeilded a solution for me. Im not even sure what
i am trying
is even possible! Some sort of clarification on this would really help!

Cheers

Nick...




On 29 May 2013 15:57, Andy Lester a...@petdance.com wrote:


 On May 29, 2013, at 9:46 AM, Dyer, James james.d...@ingramcontent.com
 wrote:

  Just an instanity check, I see I had misspelled maxCollations as
 maxCollation in my prior response.  When you tested with this set the
 same as maxCollationTries, did you correct my spelling?

 Yes, definitely.

 Thanks for the ticket.  I am looking at the effects of turning on
 spellcheck.onlyMorePopular to true, which reduces the number of collations
 it seems to do, but doesn't affect the underlying question of is the
 spellchecker doing FQs properly?

 Thanks,
 Andy

 --
 Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance




-- 
Nick Fellows
DJdownload.com
---
10 Greenland Street
London
NW10ND
United Kingdom
---
n...@djdownload.com (E)

---
www.djdownload.com

Re: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Shalin Shekhar Mangar

James, this is very useful information. Can you please add this to the wiki?


On Wed, May 29, 2013 at 10:36 PM, Dyer, James
james.d...@ingramcontent.comwrote:

 Instead of maxCollationTries=0, use a value greater than zero.  Zero
 means not to check if the collation will return hits.  1 means to test 1
 possible combination against the index and return it only if it returns
 hits.  2 tries up to 2 possibilities, etc.  As you have
 spellcheck.maxCollations=8, you'll probably want maxCollationTries at
 least that large.  Maybe 10-20 would be better.  Make it as low as possible
 to get generally good results, or as high as possible before the
 performance on a query with many misspelled words gets too bad.

 Also, use a spellcheck.count greater than 2.  This is as many corrections
 per misspelled term you want it to consider.  If using
 DirectSolrSpellChecker, you can have it set low, 5-10 might be good.  If
 using IndexBased- or FileBased spell checkers, use at least 10.

 Also, do not use onlyMorePopular unless you indeed want every term in
 the user's query to be replaced with higher-frequency terms (even
 correctly-spelled terms get replaced).  If you want it to suggest even for
 words that are in the dictionary, try spellcheck.alternativeTermCount
 instead.  Try setting it to about half of spellcheck.count (but at least
 10 if using IndexBased- or FileBased spell checkers).

 James Dyer
 Ingram Content Group
 (615) 213-4311


 -Original Message-
 From: Nicholas Fellows [mailto:n...@djdownload.com]
 Sent: Wednesday, May 29, 2013 11:06 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Why do FQs make my spelling suggestions so slow?

 I also have problems getting the solrspellchecker to utilise existing FQ
 params correctly.
 we have some fairly monster queries

 eg : http://pastebin.com/4XzGpfeC

 I cannot seem to get our FQ parameters to be honored when generating
 results.
 In essence i am getting collations that yield no results when the filter
 query is applied.

 We have items that are by default not shown when out of stock or
 forthcoming. the user
 can select whether to show these or not.

 Is there something wrong with my query or perhaps my use case is not
 supported?

 Im using nested query and local params etc

 Would very much appreciate some assistance on this one as 2days worth of
 hacking, and pestering
 people on IRC have not yet yeilded a solution for me. Im not even sure what
 i am trying
 is even possible! Some sort of clarification on this would really help!

 Cheers

 Nick...




 On 29 May 2013 15:57, Andy Lester a...@petdance.com wrote:

 
  On May 29, 2013, at 9:46 AM, Dyer, James james.d...@ingramcontent.com
 
  wrote:
 
   Just an instanity check, I see I had misspelled maxCollations as
  maxCollation in my prior response.  When you tested with this set the
  same as maxCollationTries, did you correct my spelling?
 
  Yes, definitely.
 
  Thanks for the ticket.  I am looking at the effects of turning on
  spellcheck.onlyMorePopular to true, which reduces the number of
 collations
  it seems to do, but doesn't affect the underlying question of is the
  spellchecker doing FQs properly?
 
  Thanks,
  Andy
 
  --
  Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance
 
 


 --
 Nick Fellows
 DJdownload.com
 ---
 10 Greenland Street
 London
 NW10ND
 United Kingdom
 ---
 n...@djdownload.com (E)

 ---
 www.djdownload.com




-- 
Regards,
Shalin Shekhar Mangar.

RE: Why do FQs make my spelling suggestions so slow?

2013-05-29 Thread Dyer, James

It has been in the wiki, more or less.  See 
http://wiki.apache.org/solr/SpellCheckComponent#spellcheck.count and following 
sections.

James Dyer
Ingram Content Group
(615) 213-4311


-Original Message-
From: Shalin Shekhar Mangar [mailto:shalinman...@gmail.com] 
Sent: Wednesday, May 29, 2013 12:41 PM
To: solr-user@lucene.apache.org
Subject: Re: Why do FQs make my spelling suggestions so slow?

James, this is very useful information. Can you please add this to the wiki?


On Wed, May 29, 2013 at 10:36 PM, Dyer, James
james.d...@ingramcontent.comwrote:

 Instead of maxCollationTries=0, use a value greater than zero.  Zero
 means not to check if the collation will return hits.  1 means to test 1
 possible combination against the index and return it only if it returns
 hits.  2 tries up to 2 possibilities, etc.  As you have
 spellcheck.maxCollations=8, you'll probably want maxCollationTries at
 least that large.  Maybe 10-20 would be better.  Make it as low as possible
 to get generally good results, or as high as possible before the
 performance on a query with many misspelled words gets too bad.

 Also, use a spellcheck.count greater than 2.  This is as many corrections
 per misspelled term you want it to consider.  If using
 DirectSolrSpellChecker, you can have it set low, 5-10 might be good.  If
 using IndexBased- or FileBased spell checkers, use at least 10.

 Also, do not use onlyMorePopular unless you indeed want every term in
 the user's query to be replaced with higher-frequency terms (even
 correctly-spelled terms get replaced).  If you want it to suggest even for
 words that are in the dictionary, try spellcheck.alternativeTermCount
 instead.  Try setting it to about half of spellcheck.count (but at least
 10 if using IndexBased- or FileBased spell checkers).

 James Dyer
 Ingram Content Group
 (615) 213-4311


 -Original Message-
 From: Nicholas Fellows [mailto:n...@djdownload.com]
 Sent: Wednesday, May 29, 2013 11:06 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Why do FQs make my spelling suggestions so slow?

 I also have problems getting the solrspellchecker to utilise existing FQ
 params correctly.
 we have some fairly monster queries

 eg : http://pastebin.com/4XzGpfeC

 I cannot seem to get our FQ parameters to be honored when generating
 results.
 In essence i am getting collations that yield no results when the filter
 query is applied.

 We have items that are by default not shown when out of stock or
 forthcoming. the user
 can select whether to show these or not.

 Is there something wrong with my query or perhaps my use case is not
 supported?

 Im using nested query and local params etc

 Would very much appreciate some assistance on this one as 2days worth of
 hacking, and pestering
 people on IRC have not yet yeilded a solution for me. Im not even sure what
 i am trying
 is even possible! Some sort of clarification on this would really help!

 Cheers

 Nick...




 On 29 May 2013 15:57, Andy Lester a...@petdance.com wrote:

 
  On May 29, 2013, at 9:46 AM, Dyer, James james.d...@ingramcontent.com
 
  wrote:
 
   Just an instanity check, I see I had misspelled maxCollations as
  maxCollation in my prior response.  When you tested with this set the
  same as maxCollationTries, did you correct my spelling?
 
  Yes, definitely.
 
  Thanks for the ticket.  I am looking at the effects of turning on
  spellcheck.onlyMorePopular to true, which reduces the number of
 collations
  it seems to do, but doesn't affect the underlying question of is the
  spellchecker doing FQs properly?
 
  Thanks,
  Andy
 
  --
  Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance
 
 


 --
 Nick Fellows
 DJdownload.com
 ---
 10 Greenland Street
 London
 NW10ND
 United Kingdom
 ---
 n...@djdownload.com (E)

 ---
 www.djdownload.com




-- 
Regards,
Shalin Shekhar Mangar.

Why do FQs make my spelling suggestions so slow?

2013-05-28 Thread Andy Lester

I'm working on using spellcheck for giving suggestions, and collations
are giving me good results, but they turn out to be very slow if
my original query has any FQs in it.  We can do 100 maxCollationTries
in no time at all, but if there are FQs in the query, things get
very slow.  As maxCollationTries and the count of FQs increase,
things get very slow very quickly.

 1102050   100 MaxCollationTries
0FQs 8 9101110
1FQ 11   160   599  1597  1668
2FQs20   346  1163  3360  3361
3FQs29   474  1852  5039  5095
4FQs36   589  2463  6797  6807

All times are QTimes of ms.

See that top row?  With no FQs, 50 MaxCollationTries comes back
instantly.  Add just one FQ, though, and things go bad, and they
get worse as I add more of the FQs.  Also note that things seem to
level off at 100 MaxCollationTries.

Here's a query that I've been using as a test:

df=title_tracings_t
fl=flrid,nodeid,title_tracings_t
q=bagdad+AND+diaries+AND+-parent_tracings:(bagdad+AND+diaries)
spellcheck.q=bagdad+AND+diaries
rows=4
wt=xml
sort=popular_score+desc,+grouping+asc,+copyrightyear+desc,+flrid+asc
spellcheck=true
spellcheck.dictionary=direct
spellcheck.onlyMorePopular=false
spellcheck.count=15
spellcheck.extendedResults=false
spellcheck.collate=true
spellcheck.maxCollations=10
spellcheck.maxCollationTries=50
spellcheck.collateExtendedResults=true
spellcheck.alternativeTermCount=5
spellcheck.maxResultsForSuggest=10
debugQuery=off
fq=((grouping:1+OR+grouping:2+OR+grouping:3)+OR+solrtype:N)
fq=((item_source:F+OR+item_source:B+OR+item_source:M)+OR+solrtype:N)
fq={!tag%3Dgrouping}((grouping:1+OR+grouping:2)+OR+solrtype:N)
fq={!tag%3Dlanguagecode}(languagecode:eng+OR+solrtype:N)

The only thing that changes between tests is the value of
spellcheck.maxCollationTries and how many FQs are at the end.

Am I doing something wrong?  Do the collation internals not handle
FQs correctly?  The lookup/hit counts on filterCache seem to be
increasing just fine.  It will do N lookups, N hits, so I'm not
thinking that caching is the problem.

We'd really like to be able to use the spellchecker but the results
with only 10-20 maxCollationTries aren't nearly as good as if we
can bump that up to 100, but we can't afford the slow response time.
We also can't do without the FQs.

Thanks,
Andy


--
Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance

RE: Why do FQs make my spelling suggestions so slow?

2013-05-28 Thread Dyer, James

Andy,

What are the QTimes for the 0fq,1fq,2fq,4fq  4fq cases with spellcheck 
entirely turned off?  Is it about (or a little more than) half the total when 
maxCollationTries=1 ?  Also, with the varying # of fq's, how many collation 
tries does it take to get 10 collations?

Possibly, a better way to test this is to set maxCollations = 
maxCollationTries.  The reason is that it quits trying once it finds 
maxCollations, so if with 0fq's, lots of combinations can generate hits and 
it doesn't need to try very many to get to 10.  But with more fq's, fewer 
collations will pan out so now it is trying more up to 100 before (if ever) it 
gets to 10.

I would predict that for each try it has to do (and you can force this by 
setting maxCollations = maxCollationTries), qtime will grow linerally per try.  
(I'm assuming you have all non-search components like faceting turned off).  So 
say with 2fq's it takes 10ms for the query to complete with spellcheck off, and 
20ms with maxCollation = maxCollationTries = 1, then it will take about 110ms 
with maxCollation = maxCollationTries = 10.

Now if you are finding that with a certain # of fq's, qtime with spellcheck off 
is, for instance, 2ms, 1 try is 10ms, 2 tries is 19ms, etc, then this is more 
than liner growth.  In this case we would need to look at how spell check 
applies fq's and see if there is a bug with it using the cache correctly.

But I think you're just setting maxCollationTries too high.  You're asking it 
to do too much work in trying teens of combinations.  Really, this feature was 
designed to spellcheck and not suggest.  But see 
https://issues.apache.org/jira/browse/SOLR-3240 , which is committed to the 4x 
branch for inclusion in an eventual 4.4 release.  This will make the time to do 
collation tries growth less than linear, possibly making it more suitable for 
suggest.

James Dyer
Ingram Content Group
(615) 213-4311


-Original Message-
From: Andy Lester [mailto:a...@petdance.com] 
Sent: Tuesday, May 28, 2013 2:29 PM
To: solr-user@lucene.apache.org
Subject: Why do FQs make my spelling suggestions so slow?

I'm working on using spellcheck for giving suggestions, and collations
are giving me good results, but they turn out to be very slow if
my original query has any FQs in it.  We can do 100 maxCollationTries
in no time at all, but if there are FQs in the query, things get
very slow.  As maxCollationTries and the count of FQs increase,
things get very slow very quickly.

 1102050   100 MaxCollationTries
0FQs 8 9101110
1FQ 11   160   599  1597  1668
2FQs20   346  1163  3360  3361
3FQs29   474  1852  5039  5095
4FQs36   589  2463  6797  6807

All times are QTimes of ms.

See that top row?  With no FQs, 50 MaxCollationTries comes back
instantly.  Add just one FQ, though, and things go bad, and they
get worse as I add more of the FQs.  Also note that things seem to
level off at 100 MaxCollationTries.

Here's a query that I've been using as a test:

df=title_tracings_t
fl=flrid,nodeid,title_tracings_t
q=bagdad+AND+diaries+AND+-parent_tracings:(bagdad+AND+diaries)
spellcheck.q=bagdad+AND+diaries
rows=4
wt=xml
sort=popular_score+desc,+grouping+asc,+copyrightyear+desc,+flrid+asc
spellcheck=true
spellcheck.dictionary=direct
spellcheck.onlyMorePopular=false
spellcheck.count=15
spellcheck.extendedResults=false
spellcheck.collate=true
spellcheck.maxCollations=10
spellcheck.maxCollationTries=50
spellcheck.collateExtendedResults=true
spellcheck.alternativeTermCount=5
spellcheck.maxResultsForSuggest=10
debugQuery=off
fq=((grouping:1+OR+grouping:2+OR+grouping:3)+OR+solrtype:N)
fq=((item_source:F+OR+item_source:B+OR+item_source:M)+OR+solrtype:N)
fq={!tag%3Dgrouping}((grouping:1+OR+grouping:2)+OR+solrtype:N)
fq={!tag%3Dlanguagecode}(languagecode:eng+OR+solrtype:N)

The only thing that changes between tests is the value of
spellcheck.maxCollationTries and how many FQs are at the end.

Am I doing something wrong?  Do the collation internals not handle
FQs correctly?  The lookup/hit counts on filterCache seem to be
increasing just fine.  It will do N lookups, N hits, so I'm not
thinking that caching is the problem.

We'd really like to be able to use the spellchecker but the results
with only 10-20 maxCollationTries aren't nearly as good as if we
can bump that up to 100, but we can't afford the slow response time.
We also can't do without the FQs.

Thanks,
Andy


--
Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance

Re: Why do FQs make my spelling suggestions so slow?

2013-05-28 Thread Andy Lester

Thanks for looking at this.

 What are the QTimes for the 0fq,1fq,2fq,4fq  4fq cases with spellcheck 
 entirely turned off?  Is it about (or a little more than) half the total when 
 maxCollationTries=1 ?

With spellcheck off I get 8ms for 4fq query.


  Also, with the varying # of fq's, how many collation tries does it take to 
 get 10 collations?

I don't know.  How can I tell?


 Possibly, a better way to test this is to set maxCollations = 
 maxCollationTries.  The reason is that it quits trying once it finds 
 maxCollations, so if with 0fq's, lots of combinations can generate hits and 
 it doesn't need to try very many to get to 10.  But with more fq's, fewer 
 collations will pan out so now it is trying more up to 100 before (if ever) 
 it gets to 10.

It does just fine doing 100 collations so long as there are no FQs.  It seems 
to me that the FQs are taking an inordinate amount of extra time.  100 
collations in (roughly) the same amount of time as a single collation, so long 
as there are no FQs.  Why are the FQs such a drag on the collation process?


 (I'm assuming you have all non-search components like faceting turned off).

Yes, definitely.


  So say with 2fq's it takes 10ms for the query to complete with spellcheck 
 off, and 20ms with maxCollation = maxCollationTries = 1, then it will take 
 about 110ms with maxCollation = maxCollationTries = 10.

I can do maxCollation = maxCollationTries = 100 and it comes back in 14ms, so 
long as I have FQs off.  Add a single FQ and it becomes 13499ms.

I can do maxCollation = maxCollationTries = 1000 and it comes back in 45ms, so 
long as I have FQs off.  Add a single FQ and it becomes 62038ms.


 But I think you're just setting maxCollationTries too high.  You're asking it 
 to do too much work in trying teens of combinations.

The results I get back with 100 tries are about twice as many as I get with 10 
tries.  That's a big difference to the user where it's trying to figure 
misspelled phrases.

Andy

--
Andy Lester = a...@petdance.com = www.petdance.com = AIM:petdance

Re: Context-Sensitive Spelling Suggestions Collations

2011-09-02 Thread O. Klein

Cool, I will check it out.

Are these changes present in nightly build? Or do I have to make my own
build?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3303941.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Context-Sensitive Spelling Suggestions Collations

2011-09-02 Thread Dyer, James

You will need get the source, apply the patch and build solr for yourself.  
There are some instructions at http://wiki.apache.org/solr/HowToContribute .  
It would be great if you could try this out and provide feedback.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311


-Original Message-
From: O. Klein [mailto:kl...@octoweb.nl] 
Sent: Friday, September 02, 2011 6:22 AM
To: solr-user@lucene.apache.org
Subject: Re: Context-Sensitive Spelling Suggestions  Collations

Cool, I will check it out.

Are these changes present in nightly build? Or do I have to make my own
build?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3303941.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Context-Sensitive Spelling Suggestions Collations

2011-09-02 Thread O. Klein

OK, I can confirm that the spellchecker now has the correct behaviour.

Eventhough a misspelled word is found in the index, it now says it is
correctlySpelled=false and gives the proper suggestion.

Thanx a bunch!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3304552.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Context-Sensitive Spelling Suggestions Collations

2011-09-02 Thread O. Klein

When searching for a misspelled word that is in index and a misspelled word
that isn't, the collation doesn't use the suggestion for word that is
misspelled in the index.

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3304888.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Context-Sensitive Spelling Suggestions Collations

2011-09-01 Thread Dyer, James

The changes to DirectSpellChecker are included in SOLR-2585 patch, which I 
sync'ed to the current Trunk today.  So all you have to do is apply the patch, 
build and then add the 1-2 new parameters to your query:

- spellcheck.alternativeTermCount - the # of suggestions you want to generate 
on terms that were found in the index (set greater to 0).

- spellcheck.maxResultsForSuggest - the max # of query results to disable 
spellingcheck (normally you'd want 0 for regular spellchecking and something 0 
to generate did you mean suggestions) ... 0 is the default.

If you do try this patch out, please let us know how it works out and also any 
suggestions you have for improving it.  Solr's current inability to consider 
fixes for words appearing in the index is to me a significant functionality 
gap.  I would love to see something like this eventually added to the product 
so any feedback you can give is appreciated.  Thanks.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311


-Original Message-
From: O. Klein [mailto:kl...@octoweb.nl] 
Sent: Tuesday, August 30, 2011 8:45 AM
To: solr-user@lucene.apache.org
Subject: Context-Sensitive Spelling Suggestions  Collations

Using the DirectSolrSpellChecker im very interested in this.

According to https://issues.apache.org/jira/browse/SOLR-2585 some changes
need to be made to DirectSolrSpellChecker.

Does anybody know how to get this working?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3295570.html
Sent from the Solr - User mailing list archive at Nabble.com.

Context-Sensitive Spelling Suggestions Collations

2011-08-30 Thread O. Klein

Using the DirectSolrSpellChecker im very interested in this.

According to https://issues.apache.org/jira/browse/SOLR-2585 some changes
need to be made to DirectSolrSpellChecker.

Does anybody know how to get this working?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3295570.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling Suggestions vs Correction

2011-01-01 Thread TxCSguy




Thanks for the Reply Erick.  I read through the link you provided as well as 
http://wiki.apache.org/solr/TermsComponent
http://wiki.apache.org/solr/TermsComponent  but that doesn't seem to be what
I'm looking for.  If a user searches for pell, is there a way to have SOLR
modify the query such that the highest-ranked spell correction (dell in
this example) is search for by the query component rather than pell. Right
now, it seems that all that is happening is that dell is being returned as
a suggestion, but the query component is still searching with pell.

Thanks in advance!

-Mark 




-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2176048.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling Suggestions vs Correction

2011-01-01 Thread François Schiettecatte

I have just been playing around with spell check and from what I can tell it 
does not do that automatically, you would need to program that in your 
application yourself, basically run a search, check the spellcheck, run the 
search again if needed, and present the results to the user, not all that 
complicated and you could do that recursively since you would not get a 
spellcheck back on the second pass.

Cheers

François

On Jan 1, 2011, at 9:23 AM, TxCSguy wrote:

 
 
 
 Thanks for the Reply Erick.  I read through the link you provided as well as 
 http://wiki.apache.org/solr/TermsComponent
 http://wiki.apache.org/solr/TermsComponent  but that doesn't seem to be what
 I'm looking for.  If a user searches for pell, is there a way to have SOLR
 modify the query such that the highest-ranked spell correction (dell in
 this example) is search for by the query component rather than pell. Right
 now, it seems that all that is happening is that dell is being returned as
 a suggestion, but the query component is still searching with pell.
 
 Thanks in advance!
 
 -Mark 
 
 
 
 
 -- 
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2176048.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling Suggestions vs Correction

2011-01-01 Thread Erick Erickson

Ah, didn't tumble to that

Not that I know of. I suspect that the way to handle this would be
to catch the response and inspect it and then fire off a second
query yourself in the middleware layer. That gives you the chance
to form the response page that the user sees and clearly show
that that you did something other than what they requested, even
though what they requested was nonsense.

Best
Erick

On Sat, Jan 1, 2011 at 9:23 AM, TxCSguy markefonte...@gmail.com wrote:




 Thanks for the Reply Erick.  I read through the link you provided as well
 as
 http://wiki.apache.org/solr/TermsComponent
 http://wiki.apache.org/solr/TermsComponent  but that doesn't seem to be
 what
 I'm looking for.  If a user searches for pell, is there a way to have
 SOLR
 modify the query such that the highest-ranked spell correction (dell in
 this example) is search for by the query component rather than pell.
 Right
 now, it seems that all that is happening is that dell is being returned
 as
 a suggestion, but the query component is still searching with pell.

 Thanks in advance!

 -Mark




 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2176048.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Spelling Suggestions vs Correction

2010-12-31 Thread TxCSguy


Hi,

I am trying to clear up some confusion about SOLR's spell check
functionality.  Being new to SORL and Lucene as well, I was under the
assumption that spellcheck would take a query entered by a user and end up
actually querying the index based upon the corrections returned by the
spellcheck component.  

For example (referring to the sample data that comes with SOLR),  let's say
that the user issues a search for sell when what they really are looking
for is dell.  So the spellcheck component is returning the suggestion of
dell (I can see this in the response) but still searching the index for
sell.

So my questions are:
1) Is my example above a correct representation of what is happening by
incorporating the spellcheck component in?
2) Is there a way to actually search for the corrected version (dell) rather
than the original version (sell)?

Thanks in advance for any help,
Mark 
  


-- 
View this message in context: 
http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2172572.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Spelling Suggestions vs Correction

2010-12-31 Thread Erick Erickson

Here's a discussion of the difference between them, does that answer?

http://lucene.472066.n3.nabble.com/spell-check-vs-terms-component-td1870214.html

Best
Erick

On Fri, Dec 31, 2010 at 8:55 AM, TxCSguy markefonte...@gmail.com wrote:


 Hi,

 I am trying to clear up some confusion about SOLR's spell check
 functionality.  Being new to SORL and Lucene as well, I was under the
 assumption that spellcheck would take a query entered by a user and end up
 actually querying the index based upon the corrections returned by the
 spellcheck component.

 For example (referring to the sample data that comes with SOLR),  let's say
 that the user issues a search for sell when what they really are looking
 for is dell.  So the spellcheck component is returning the suggestion of
 dell (I can see this in the response) but still searching the index for
 sell.

 So my questions are:
 1) Is my example above a correct representation of what is happening by
 incorporating the spellcheck component in?
 2) Is there a way to actually search for the corrected version (dell)
 rather
 than the original version (sell)?

 Thanks in advance for any help,
 Mark



 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Spelling-Suggestions-vs-Correction-tp2172572p2172572.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: Query regarding Spelling Suggestions

2009-01-13 Thread Deshpande, Mukta

Hi Grant,

My spellcheck is now working fine with the following configuration:

searchComponent name=spellcheck class=solr.SpellCheckComponent
   lst name=spellchecker
  str name=fieldTypeword/str
  str name=classnamesolr.IndexBasedSpellChecker/str
  str name=fieldword/str
  str name=characterEncodingUTF-8/str
  str
name=sourceLocationd:\solr-tomcat\solr\data\syn_index/str
  str name=spellcheckIndexDir./spellcheckerFile1/str
   /lst
 /searchComponent

Earlier I configured the lucene-index (dictionary) syn_index to the
spellcheckIndexDir as interpreted from the wiki page.
Then I was looking into the file IndexBasedSpellChecker.java and found
the usage of sourceLocation. 
When I configured my lucene-index (dictionary) syn_index as
sourceLocation the IndexBasedSpellChecker worked.

I have following question / observation : (just to ensure that my
configurations are correct)

The lucene-index (dictionary) syn_index is already an index so do we
have to specify the spellcheckIndexDir again?
(If I do not give the spellcheckIndexDir I do not get any
suggestions.)
When I give the build command the spellcheckIndexDir gets populated
reading the syn_index. Can we avoid this duplication?

If the sourceLocation is mandatory when using a third party index for
spelling suggestions, may I update the Solr WIKI to include this
important information.

Thanks  Best Regards,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org] 
Sent: Monday, January 12, 2009 10:15 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Solr 1.3 doesn't use Log4J, it uses Java Utility Logging (JUL).  I
believe the info level in the logs is sufficient.  Let's start by
posting what you have?

Also, are you able to get the sample spellchecking to work?

On Jan 12, 2009, at 2:16 AM, Deshpande, Mukta wrote:

 Hi,

 Could you please send me the needful entries in log4j.properties to 
 enable logging, explicitly for SpellCheckComponent.

 My current log4j.properties looks like:

 log4j.rootLogger=INFO,console
 log4j.appender.console=org.apache.log4j.ConsoleAppender
 log4j.appender.console.target=System.err
 log4j.appender.console.layout=org.apache.log4j.PatternLayout
 log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd
 HH:mm:ss} %p
 %c{2}: %m%n
 log4j.logger.org.apache.solr=DEBUG

 With these settings I can only see the INFO level logs.

 I tried to change the log level for SpellCheckComponent to FINE  
 using
 the admin logging page http://localhost:8080/solr/admin/logging but 
 did not see any difference in logging.

 Thanks,
 ~Mukta

 -Original Message-
 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Monday, January 12, 2009 3:22 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions

 Can you send the full log?

 On Jan 11, 2009, at 1:51 PM, Deshpande, Mukta wrote:

 I am using the example schema that comes with the Solr installation 
 downloaded from http://www.mirrorgeek.com/apache.org/lucene/solr/.
 I have added the word  field with textSpell fieldtype in the 
 schema.xml file, as specified in the below mail.

 My spelling index exist under SOLR HOME/data/ If I open my index in

 Luke  I can see the entries against word
 field.

 Thanks,
 ~Mukta


 

 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Fri 1/9/2009 8:29 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions



 Can you put the full log (as short as possibly demonstrates the
 problem) somewhere where I can take a look?  Likewise, can you share 
 your schema?

 Also, does the spelling index exist under SOLR HOME/data/index?  If

 you open it w/ Luke, does it have entries?

 Thanks,
 Grant

 On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:


 Yes. I send the build command as:
 http://localhost:8080/solr/select/? 
 q=documnetspellcheck=truespellch
 eck
 .build
 =truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
 y=dict

 The Tomcat log shows:
 Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
 INFO: [] webapp=/solr path=/select/
 params
 ={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
 onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
 QTime=141

 Even after sending the build command I do not get any suggestions.
 Can you please check.

 Thanks,
 ~Mukta

 -Original Message-
 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Thursday, January 08, 2009 7:42 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions

 Did you send in the build command?  See 
 http://wiki.apache.org/solr/SpellCheckComponent

 On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:

 Hi,

 I am using Wordnet dictionary for spelling suggestions.

 The dictionary is converted to Solr index  with only one field 
 word
 and stored in location solr-home/data/syn_index, using 
 syns2Index.java

Re: Query regarding Spelling Suggestions

2009-01-13 Thread Shalin Shekhar Mangar

On Tue, Jan 13, 2009 at 5:16 PM, Deshpande, Mukta mudes...@ptc.com wrote:

 I have following question / observation : (just to ensure that my
 configurations are correct)

 The lucene-index (dictionary) syn_index is already an index so do we
 have to specify the spellcheckIndexDir again?
(If I do not give the spellcheckIndexDir I do not get any
 suggestions.)


The syn_index here is your Lucene index from which you want to use as the
source for words. Spell checker processes each token to create n-grams which
are then stored into a lucene index at the spellCheckIndexDir or in
memory. This is why you need to specify both sourceLocation and
spellcheckIndexDir.

If you do not give spellCheckIndexDir, spell checker will create a Lucene
index in-memory, so it should still work. Are you sure you gave a build
command before issuing the query?


 When I give the build command the spellcheckIndexDir gets populated
 reading the syn_index. Can we avoid this duplication?


Spell checker needs a Lucene index to work. It creates a new one and adds
tokens after some processing to this index. There is no way to avoid
creation of another index at present.

However, it should be possible to modify it to store it's fields inside an
existing Lucene index (maybe even Solr's own index). Contributions are
always welcome :)


 If the sourceLocation is mandatory when using a third party index for
 spelling suggestions, may I update the Solr WIKI to include this
 important information.


Sure, please go ahead. Thanks!

-- 
Regards,
Shalin Shekhar Mangar.

RE: Query regarding Spelling Suggestions

2009-01-13 Thread Deshpande, Mukta

Thanks all for the help and information.

Best Regards,
~Mukta

-Original Message-
From: Shalin Shekhar Mangar [mailto:shalinman...@gmail.com] 
Sent: Tuesday, January 13, 2009 6:50 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

On Tue, Jan 13, 2009 at 5:16 PM, Deshpande, Mukta mudes...@ptc.com
wrote:

 I have following question / observation : (just to ensure that my 
 configurations are correct)

 The lucene-index (dictionary) syn_index is already an index so do we

 have to specify the spellcheckIndexDir again?
(If I do not give the spellcheckIndexDir I do not get any
 suggestions.)

The syn_index here is your Lucene index from which you want to use as
the source for words. Spell checker processes each token to create
n-grams which are then stored into a lucene index at the
spellCheckIndexDir or in memory. This is why you need to specify both
sourceLocation and spellcheckIndexDir.

If you do not give spellCheckIndexDir, spell checker will create a
Lucene index in-memory, so it should still work. Are you sure you gave a
build command before issuing the query?

 When I give the build command the spellcheckIndexDir gets populated 
 reading the syn_index. Can we avoid this duplication?

Spell checker needs a Lucene index to work. It creates a new one and
adds tokens after some processing to this index. There is no way to
avoid creation of another index at present.

However, it should be possible to modify it to store it's fields inside
an existing Lucene index (maybe even Solr's own index). Contributions
are always welcome :)

 If the sourceLocation is mandatory when using a third party index 
 for spelling suggestions, may I update the Solr WIKI to include this 
 important information.

Sure, please go ahead. Thanks!

--
Regards,
Shalin Shekhar Mangar.

Re: Query regarding Spelling Suggestions

2009-01-12 Thread Grant Ingersoll

Solr 1.3 doesn't use Log4J, it uses Java Utility Logging (JUL).  I  
believe the info level in the logs is sufficient.  Let's start by  
posting what you have?


Also, are you able to get the sample spellchecking to work?

On Jan 12, 2009, at 2:16 AM, Deshpande, Mukta wrote:


Hi,

Could you please send me the needful entries in log4j.properties to
enable logging, explicitly for SpellCheckComponent.

My current log4j.properties looks like:

log4j.rootLogger=INFO,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd  
HH:mm:ss} %p

%c{2}: %m%n
log4j.logger.org.apache.solr=DEBUG

With these settings I can only see the INFO level logs.

I tried to change the log level for SpellCheckComponent to FINE  
using
the admin logging page http://localhost:8080/solr/admin/logging but  
did

not see any difference in logging.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Monday, January 12, 2009 3:22 AM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Can you send the full log?

On Jan 11, 2009, at 1:51 PM, Deshpande, Mukta wrote:


I am using the example schema that comes with the Solr installation
downloaded from http://www.mirrorgeek.com/apache.org/lucene/solr/.
I have added the word  field with textSpell fieldtype in the
schema.xml file, as specified in the below mail.

My spelling index exist under SOLR HOME/data/ If I open my index in
Luke  I can see the entries against word
field.

Thanks,
~Mukta




From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Fri 1/9/2009 8:29 AM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions



Can you put the full log (as short as possibly demonstrates the
problem) somewhere where I can take a look?  Likewise, can you share
your schema?

Also, does the spelling index exist under SOLR HOME/data/index?  If
you open it w/ Luke, does it have entries?

Thanks,
Grant

On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:



Yes. I send the build command as:
http://localhost:8080/solr/select/? 
q=documnetspellcheck=truespellch

eck
.build
=truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
y=dict

The Tomcat log shows:
Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select/
params
={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
QTime=141

Even after sending the build command I do not get any suggestions.
Can you please check.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Thursday, January 08, 2009 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Did you send in the build command?  See
http://wiki.apache.org/solr/SpellCheckComponent

On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:


Hi,

I am using Wordnet dictionary for spelling suggestions.

The dictionary is converted to Solr index  with only one field
word
and stored in location solr-home/data/syn_index, using
syns2Index.java program available at
http://www.tropo.com/techno/java/lucene/wordnet.html

I have added the word field in my schema.xml as field
name=word



type=textSpell indexed=true stored=true/

My application data indexes are in solr-home/data

I am trying to use solr.IndexBasedSpellChecker to get spelling
suggestions.

My spell check component is configured as:

searchComponent name=spellcheck  
class=solr.SpellCheckComponent



str name=queryAnalyzerFieldTypetextSpell/str
lst name=spellchecker
  str name=namedict/str
  str name=classnamesolr.IndexBasedSpellChecker/str
  str name=fieldword/str
  str name=characterEncodingUTF-8/str
  str name=spellcheckIndexDir./syn_index/str
/lst
/searchComponent

I have added this component to my standard request handler as:

requestHandler name=standard class=solr.StandardRequestHandler
default=true
 lst name=defaults
 str name=echoParamsexplicit/str
 /lst
 arr name=last-components
 strspellcheck/str
 /arr
/requestHandler

With the above configuration, I do not get any spelling  
suggestions.

Can
somebody help ASAP.

Thanks,
~Mukta













--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

RE: Query regarding Spelling Suggestions

2009-01-11 Thread Deshpande, Mukta

I am using the example schema that comes with the Solr installation downloaded 
from http://www.mirrorgeek.com/apache.org/lucene/solr/.
I have added the word  field with textSpell fieldtype in the schema.xml 
file, as specified in the below mail.

My spelling index exist under SOLR HOME/data/
If I open my index in Luke  I can see the entries against word field.

Thanks,
~Mukta




From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Fri 1/9/2009 8:29 AM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions



Can you put the full log (as short as possibly demonstrates the 
problem) somewhere where I can take a look?  Likewise, can you share 
your schema?

Also, does the spelling index exist under SOLR HOME/data/index?  If 
you open it w/ Luke, does it have entries?

Thanks,
Grant

On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:


 Yes. I send the build command as:
 http://localhost:8080/solr/select/?q=documnetspellcheck=truespellcheck
 .build
 =truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
 y=dict

 The Tomcat log shows:
 Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
 INFO: [] webapp=/solr path=/select/
 params
 ={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
 onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
 QTime=141

 Even after sending the build command I do not get any suggestions.
 Can you please check.

 Thanks,
 ~Mukta

 -Original Message-
 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Thursday, January 08, 2009 7:42 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions

 Did you send in the build command?  See
 http://wiki.apache.org/solr/SpellCheckComponent

 On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:

 Hi,

 I am using Wordnet dictionary for spelling suggestions.

 The dictionary is converted to Solr index  with only one field word
 and stored in location solr-home/data/syn_index, using
 syns2Index.java program available at
 http://www.tropo.com/techno/java/lucene/wordnet.html

 I have added the word field in my schema.xml as field 
 name=word

 type=textSpell indexed=true stored=true/

 My application data indexes are in solr-home/data

 I am trying to use solr.IndexBasedSpellChecker to get spelling
 suggestions.

 My spell check component is configured as:

 searchComponent name=spellcheck class=solr.SpellCheckComponent
  str name=queryAnalyzerFieldTypetextSpell/str
  lst name=spellchecker
 str name=namedict/str
 str name=classnamesolr.IndexBasedSpellChecker/str
 str name=fieldword/str
 str name=characterEncodingUTF-8/str
 str name=spellcheckIndexDir./syn_index/str
  /lst
 /searchComponent

 I have added this component to my standard request handler as:

 requestHandler name=standard class=solr.StandardRequestHandler
 default=true
lst name=defaults
str name=echoParamsexplicit/str
/lst
arr name=last-components
strspellcheck/str
/arr
 /requestHandler

 With the above configuration, I do not get any spelling suggestions.
 Can
 somebody help ASAP.

 Thanks,
 ~Mukta

 --
 Grant Ingersoll

 Lucene Helpful Hints:
 http://wiki.apache.org/lucene-java/BasicsOfPerformance
 http://wiki.apache.org/lucene-java/LuceneFAQ











--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

Re: Query regarding Spelling Suggestions

2009-01-11 Thread Grant Ingersoll


Can you send the full log?

On Jan 11, 2009, at 1:51 PM, Deshpande, Mukta wrote:

I am using the example schema that comes with the Solr installation  
downloaded from http://www.mirrorgeek.com/apache.org/lucene/solr/.
I have added the word  field with textSpell fieldtype in the  
schema.xml file, as specified in the below mail.


My spelling index exist under SOLR HOME/data/
If I open my index in Luke  I can see the entries against word  
field.


Thanks,
~Mukta




From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Fri 1/9/2009 8:29 AM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions



Can you put the full log (as short as possibly demonstrates the
problem) somewhere where I can take a look?  Likewise, can you share
your schema?

Also, does the spelling index exist under SOLR HOME/data/index?  If
you open it w/ Luke, does it have entries?

Thanks,
Grant

On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:



Yes. I send the build command as:
http://localhost:8080/solr/select/?q=documnetspellcheck=truespellcheck
.build
=truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
y=dict

The Tomcat log shows:
Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select/
params
={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
QTime=141

Even after sending the build command I do not get any suggestions.
Can you please check.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Thursday, January 08, 2009 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Did you send in the build command?  See
http://wiki.apache.org/solr/SpellCheckComponent

On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:


Hi,

I am using Wordnet dictionary for spelling suggestions.

The dictionary is converted to Solr index  with only one field  
word

and stored in location solr-home/data/syn_index, using
syns2Index.java program available at
http://www.tropo.com/techno/java/lucene/wordnet.html

I have added the word field in my schema.xml as field
name=word



type=textSpell indexed=true stored=true/

My application data indexes are in solr-home/data

I am trying to use solr.IndexBasedSpellChecker to get spelling
suggestions.

My spell check component is configured as:

searchComponent name=spellcheck class=solr.SpellCheckComponent
str name=queryAnalyzerFieldTypetextSpell/str
lst name=spellchecker
   str name=namedict/str
   str name=classnamesolr.IndexBasedSpellChecker/str
   str name=fieldword/str
   str name=characterEncodingUTF-8/str
   str name=spellcheckIndexDir./syn_index/str
/lst
/searchComponent

I have added this component to my standard request handler as:

requestHandler name=standard class=solr.StandardRequestHandler
default=true
  lst name=defaults
  str name=echoParamsexplicit/str
  /lst
  arr name=last-components
  strspellcheck/str
  /arr
/requestHandler

With the above configuration, I do not get any spelling suggestions.
Can
somebody help ASAP.

Thanks,
~Mukta

RE: Query regarding Spelling Suggestions

2009-01-11 Thread Deshpande, Mukta

Hi,

Could you please send me the needful entries in log4j.properties to
enable logging, explicitly for SpellCheckComponent.

My current log4j.properties looks like:

log4j.rootLogger=INFO,console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p
%c{2}: %m%n
log4j.logger.org.apache.solr=DEBUG

With these settings I can only see the INFO level logs.

I tried to change the log level for SpellCheckComponent to FINE using
the admin logging page http://localhost:8080/solr/admin/logging but did
not see any difference in logging.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org] 
Sent: Monday, January 12, 2009 3:22 AM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Can you send the full log?

On Jan 11, 2009, at 1:51 PM, Deshpande, Mukta wrote:

 I am using the example schema that comes with the Solr installation 
 downloaded from http://www.mirrorgeek.com/apache.org/lucene/solr/.
 I have added the word  field with textSpell fieldtype in the 
 schema.xml file, as specified in the below mail.

 My spelling index exist under SOLR HOME/data/ If I open my index in 
 Luke  I can see the entries against word
 field.

 Thanks,
 ~Mukta


 

 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Fri 1/9/2009 8:29 AM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions



 Can you put the full log (as short as possibly demonstrates the
 problem) somewhere where I can take a look?  Likewise, can you share 
 your schema?

 Also, does the spelling index exist under SOLR HOME/data/index?  If 
 you open it w/ Luke, does it have entries?

 Thanks,
 Grant

 On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:


 Yes. I send the build command as:
 http://localhost:8080/solr/select/?q=documnetspellcheck=truespellch
 eck
 .build
 =truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
 y=dict

 The Tomcat log shows:
 Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
 INFO: [] webapp=/solr path=/select/
 params
 ={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
 onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
 QTime=141

 Even after sending the build command I do not get any suggestions.
 Can you please check.

 Thanks,
 ~Mukta

 -Original Message-
 From: Grant Ingersoll [mailto:gsing...@apache.org]
 Sent: Thursday, January 08, 2009 7:42 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Query regarding Spelling Suggestions

 Did you send in the build command?  See 
 http://wiki.apache.org/solr/SpellCheckComponent

 On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:

 Hi,

 I am using Wordnet dictionary for spelling suggestions.

 The dictionary is converted to Solr index  with only one field 
 word
 and stored in location solr-home/data/syn_index, using 
 syns2Index.java program available at 
 http://www.tropo.com/techno/java/lucene/wordnet.html

 I have added the word field in my schema.xml as field 
 name=word

 type=textSpell indexed=true stored=true/

 My application data indexes are in solr-home/data

 I am trying to use solr.IndexBasedSpellChecker to get spelling 
 suggestions.

 My spell check component is configured as:

 searchComponent name=spellcheck class=solr.SpellCheckComponent

 str name=queryAnalyzerFieldTypetextSpell/str
 lst name=spellchecker
str name=namedict/str
str name=classnamesolr.IndexBasedSpellChecker/str
str name=fieldword/str
str name=characterEncodingUTF-8/str
str name=spellcheckIndexDir./syn_index/str
 /lst
 /searchComponent

 I have added this component to my standard request handler as:

 requestHandler name=standard class=solr.StandardRequestHandler
 default=true
   lst name=defaults
   str name=echoParamsexplicit/str
   /lst
   arr name=last-components
   strspellcheck/str
   /arr
 /requestHandler

 With the above configuration, I do not get any spelling suggestions.
 Can
 somebody help ASAP.

 Thanks,
 ~Mukta

Re: Query regarding Spelling Suggestions

2009-01-09 Thread Grant Ingersoll

Can you put the full log (as short as possibly demonstrates the  
problem) somewhere where I can take a look?  Likewise, can you share  
your schema?


Also, does the spelling index exist under SOLR HOME/data/index?  If  
you open it w/ Luke, does it have entries?


Thanks,
Grant

On Jan 8, 2009, at 11:30 PM, Deshpande, Mukta wrote:



Yes. I send the build command as:
http://localhost:8080/solr/select/?q=documnetspellcheck=truespellcheck
.build 
=truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar

y=dict

The Tomcat log shows:
Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select/
params 
={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti

onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
QTime=141

Even after sending the build command I do not get any suggestions.
Can you please check.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org]
Sent: Thursday, January 08, 2009 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Did you send in the build command?  See
http://wiki.apache.org/solr/SpellCheckComponent

On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:


Hi,

I am using Wordnet dictionary for spelling suggestions.

The dictionary is converted to Solr index  with only one field word
and stored in location solr-home/data/syn_index, using
syns2Index.java program available at
http://www.tropo.com/techno/java/lucene/wordnet.html

I have added the word field in my schema.xml as field  
name=word



type=textSpell indexed=true stored=true/

My application data indexes are in solr-home/data

I am trying to use solr.IndexBasedSpellChecker to get spelling
suggestions.

My spell check component is configured as:

searchComponent name=spellcheck class=solr.SpellCheckComponent
 str name=queryAnalyzerFieldTypetextSpell/str
 lst name=spellchecker
str name=namedict/str
str name=classnamesolr.IndexBasedSpellChecker/str
str name=fieldword/str
str name=characterEncodingUTF-8/str
str name=spellcheckIndexDir./syn_index/str
 /lst
/searchComponent

I have added this component to my standard request handler as:

requestHandler name=standard class=solr.StandardRequestHandler
default=true
   lst name=defaults
   str name=echoParamsexplicit/str
   /lst
   arr name=last-components
   strspellcheck/str
   /arr
/requestHandler

With the above configuration, I do not get any spelling suggestions.
Can
somebody help ASAP.

Thanks,
~Mukta


--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ












--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

Query regarding Spelling Suggestions

2009-01-08 Thread Deshpande, Mukta

Hi,
 
I am using Wordnet dictionary for spelling suggestions.
 
The dictionary is converted to Solr index  with only one field word
and stored in location solr-home/data/syn_index, using syns2Index.java
program available at
http://www.tropo.com/techno/java/lucene/wordnet.html
 
I have added the word field in my schema.xml as
field name=word type=textSpell indexed=true stored=true/
 
My application data indexes are in solr-home/data
 
I am trying to use solr.IndexBasedSpellChecker to get spelling
suggestions.
 
My spell check component is configured as:
 
searchComponent name=spellcheck class=solr.SpellCheckComponent
   str name=queryAnalyzerFieldTypetextSpell/str
   lst name=spellchecker
  str name=namedict/str
  str name=classnamesolr.IndexBasedSpellChecker/str
  str name=fieldword/str
  str name=characterEncodingUTF-8/str
  str name=spellcheckIndexDir./syn_index/str
   /lst
/searchComponent
 
I have added this component to my standard request handler as:
 
requestHandler name=standard class=solr.StandardRequestHandler
default=true
 lst name=defaults
 str name=echoParamsexplicit/str
 /lst
 arr name=last-components
 strspellcheck/str
 /arr
/requestHandler
 
With the above configuration, I do not get any spelling suggestions. Can
somebody help ASAP.
 
Thanks,
~Mukta

Re: Query regarding Spelling Suggestions

2009-01-08 Thread Grant Ingersoll


Did you send in the build command?  See 
http://wiki.apache.org/solr/SpellCheckComponent

On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:


Hi,

I am using Wordnet dictionary for spelling suggestions.

The dictionary is converted to Solr index  with only one field word
and stored in location solr-home/data/syn_index, using  
syns2Index.java

program available at
http://www.tropo.com/techno/java/lucene/wordnet.html

I have added the word field in my schema.xml as
field name=word type=textSpell indexed=true stored=true/

My application data indexes are in solr-home/data

I am trying to use solr.IndexBasedSpellChecker to get spelling
suggestions.

My spell check component is configured as:

searchComponent name=spellcheck class=solr.SpellCheckComponent
  str name=queryAnalyzerFieldTypetextSpell/str
  lst name=spellchecker
 str name=namedict/str
 str name=classnamesolr.IndexBasedSpellChecker/str
 str name=fieldword/str
 str name=characterEncodingUTF-8/str
 str name=spellcheckIndexDir./syn_index/str
  /lst
/searchComponent

I have added this component to my standard request handler as:

requestHandler name=standard class=solr.StandardRequestHandler
default=true
lst name=defaults
str name=echoParamsexplicit/str
/lst
arr name=last-components
strspellcheck/str
/arr
/requestHandler

With the above configuration, I do not get any spelling suggestions.  
Can

somebody help ASAP.

Thanks,
~Mukta


--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

RE: Query regarding Spelling Suggestions

2009-01-08 Thread Deshpande, Mukta


Yes. I send the build command as:
http://localhost:8080/solr/select/?q=documnetspellcheck=truespellcheck
.build=truespellcheck.count=2spellcheck.q=parfectspellcheck.dictionar
y=dict

The Tomcat log shows:
Jan 9, 2009 9:55:19 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select/
params={spellcheck=trueq=documnetspellcheck.q=parfectspellcheck.dicti
onary=dictspellcheck.count=2spellcheck.build=true} hits=0 status=0
QTime=141 

Even after sending the build command I do not get any suggestions.
Can you please check.

Thanks,
~Mukta

-Original Message-
From: Grant Ingersoll [mailto:gsing...@apache.org] 
Sent: Thursday, January 08, 2009 7:42 PM
To: solr-user@lucene.apache.org
Subject: Re: Query regarding Spelling Suggestions

Did you send in the build command?  See
http://wiki.apache.org/solr/SpellCheckComponent

On Jan 8, 2009, at 5:14 AM, Deshpande, Mukta wrote:

 Hi,

 I am using Wordnet dictionary for spelling suggestions.

 The dictionary is converted to Solr index  with only one field word
 and stored in location solr-home/data/syn_index, using 
 syns2Index.java program available at 
 http://www.tropo.com/techno/java/lucene/wordnet.html

 I have added the word field in my schema.xml as field name=word

 type=textSpell indexed=true stored=true/

 My application data indexes are in solr-home/data

 I am trying to use solr.IndexBasedSpellChecker to get spelling 
 suggestions.

 My spell check component is configured as:

 searchComponent name=spellcheck class=solr.SpellCheckComponent
   str name=queryAnalyzerFieldTypetextSpell/str
   lst name=spellchecker
  str name=namedict/str
  str name=classnamesolr.IndexBasedSpellChecker/str
  str name=fieldword/str
  str name=characterEncodingUTF-8/str
  str name=spellcheckIndexDir./syn_index/str
   /lst
 /searchComponent

 I have added this component to my standard request handler as:

 requestHandler name=standard class=solr.StandardRequestHandler
 default=true
 lst name=defaults
 str name=echoParamsexplicit/str
 /lst
 arr name=last-components
 strspellcheck/str
 /arr
 /requestHandler

 With the above configuration, I do not get any spelling suggestions.  
 Can
 somebody help ASAP.

 Thanks,
 ~Mukta

--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

Re: Query regarding spelling suggestions

2008-12-08 Thread Grant Ingersoll

Right now, you'd have to write an implementation of a  
SolrSpellChecker.  Seems like a reasonable thing to have, though.  We  
could have a Chained Spell Checker that combined the others, I think.


Another option that might work, would be to define two separate Search  
components, 1 for the file and 1 for the index, and then put them into  
the request handler, such that it might be possible to get both  
outputs.  I haven't tried this, but think it should work since the  
component just adds to the output, and doesn't overwrite it.


HTH,
Grant

On Dec 8, 2008, at 12:02 AM, Deshpande, Mukta wrote:


Hi,

I have defined a Search Component in solrconfig.xml containing two
entries for spellchecker.
One is the indexed field - title . Second is a text file -
mywords.txt.

When I enable spellchecking and query for some word, I expect that the
spelling suggestions will come from both the dictionaries. But my
observation is that the suggestions come from the indexed field only  
if

some exist otherwise I do not get any suggestions.
If I explicitly add spellcheck.dictionary=file to get suggestion from
text file then I get suggestions from the file but in that case I do  
not

get suggestions from the indexed field.

How can I get spelling suggestions from both dictionaries.

Thanks,
Mukta



--
Grant Ingersoll

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ

Re: Problem getting spelling suggestions to work

2008-05-21 Thread Chris Hostetter


: Thats true, but that's not the problem. The problem is that you can't call
: qt=spellchecker if you redefine /select in solrconfig.xml. I was wondering
: how I could add qt functionality back.

If you override /select to bind it to a specific handler, then you lose 
the abiliy to pick a handler using qt ... instead you can just map the 
SpellCheckHandler to a path (ie: name=/spellchecker instead of 
name=spellchecker)

:  Which is obviously missing the suggestions field. The reason for that is
:  likely that I overrode the default definition of /select. My /select is
:  defined in the following way:
...
:  The reason I am doing this, is that I want to replace the query component
:  with the collapse component.

it sounds like you are looking for a *component* to do spellchecking ... 
one does not exist yet (but there is an open issue where people are 
working on it ... check Jira for a patch).



-Hoss

Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy


Hello. I am having some trouble getting spelling suggestions to work. I am
running the latest nightly build of Solr. The URL I am hitting is:

http://localhost:8983/solr/select/?q=pizzzaqt=spellcheckercmd=rebuild

and the response I am getting is 

response

lst name=responseHeader
int name=status0/int
int name=QTime14/int

lst name=params
str name=cmdrebuild/str
str name=qpizzza/str
str name=qtspellchecker/str
/lst
/lst
result name=response numFound=0 start=0/
/response

Which is obviously missing the suggestions field. The reason for that is
likely that I overrode the default definition of /select. My /select is
defined in the following way:

 requestHandler name=/select
class=org.apache.solr.handler.component.SearchHandler
lst name=defaults
  str name=echoParamsexplicit/str
/lst
arr name=components
  strcollapse/str
  strfacet/str
  strmlt/str
  strhighlight/str
  strdebug/str
/arr
  /requestHandler
The reason I am doing this, is that I want to replace the query component
with the collapse component.

Am I missing something that would make the qt parameter work? Any help would
be appreciated.
-- 
View this message in context: 
http://www.nabble.com/Problem-getting-spelling-suggestions-to-work-tp17331252p17331252.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Problem getting spelling suggestions to work

2008-05-19 Thread Otis Gospodnetic

I haven't actually used this in a while, but are you asking the handler for
spellchecking (q=pizzza) or are you asking it to rebuild the index
(cmd=rebuild)? Asking for both at the same time might not be the best thing.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

- Original Message
From: oleg_gnatovskiy [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, May 19, 2008 9:01:14 PM
Subject: Problem getting spelling suggestions to work

Hello. I am having some trouble getting spelling suggestions to work. I am
running the latest nightly build of Solr. The URL I am hitting is:

http://localhost:8983/solr/select/?q=pizzzaqt=spellcheckercmd=rebuild

and the response I am getting is

0
14

rebuild
pizzza
spellchecker

Which is obviously missing the suggestions field. The reason for that is
likely that I overrode the default definition of /select. My /select is
defined in the following way:

class=org.apache.solr.handler.component.SearchHandler

explicit

collapse
facet
mlt
highlight
debug

The reason I am doing this, is that I want to replace the query component
with the collapse component.

Am I missing something that would make the qt parameter work? Any help would
be appreciated.
--
View this message in context:
http://www.nabble.com/Problem-getting-spelling-suggestions-to-work-tp17331252p17331252.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Problem getting spelling suggestions to work

2008-05-19 Thread oleg_gnatovskiy

Thats true, but that's not the problem. The problem is that you can't call
qt=spellchecker if you redefine /select in solrconfig.xml. I was wondering
how I could add qt functionality back.

Otis Gospodnetic wrote:

I haven't actually used this in a while, but are you asking the handler
for spellchecking (q=pizzza) or are you asking it to rebuild the index
(cmd=rebuild)? Asking for both at the same time might not be the best
thing.

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch

- Original Message
From: oleg_gnatovskiy [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, May 19, 2008 9:01:14 PM
Subject: Problem getting spelling suggestions to work

Hello. I am having some trouble getting spelling suggestions to work. I
am
running the latest nightly build of Solr. The URL I am hitting is:

http://localhost:8983/solr/select/?q=pizzzaqt=spellcheckercmd=rebuild

and the response I am getting is

0
14

rebuild
pizzza
spellchecker

Which is obviously missing the suggestions field. The reason for that is
likely that I overrode the default definition of /select. My /select is
defined in the following way:

class=org.apache.solr.handler.component.SearchHandler

explicit

collapse
facet
mlt
highlight
debug

The reason I am doing this, is that I want to replace the query component
with the collapse component.

Am I missing something that would make the qt parameter work? Any help
would
be appreciated.
--
View this message in context:
http://www.nabble.com/Problem-getting-spelling-suggestions-to-work-tp17331252p17331252.html
Sent from the Solr - User mailing list archive at Nabble.com.

--
View this message in context:
http://www.nabble.com/Problem-getting-spelling-suggestions-to-work-tp17331252p17333756.html
Sent from the Solr - User mailing list archive at Nabble.com.

48 matches

Mail list logo