Isn't the conclusion here that some "stopword and stemming free
matching" should be the best match if ever and to then gently degrade
to weaker forms of matching?
paul
Le 13-janv.-10 à 07:08, Walter Underwood a écrit :
There is a band named "The The". And a producer named "Don Was". For
a list of all-stopword movie titles at Netflix, see this post:
http://wunderwood.org/most_casual_observer/2007/05/invisible_titles.html
My favorite is "To Be and To Have (Être et Avoir)", which is all
stopwords in two languages. And a very good movie.
wunder
On Jan 12, 2010, at 6:55 PM, Robert Muir wrote:
sorry, i forgot to include this 2009 paper comparing what stopwords
do
across 3 languages:
http://doc.rero.ch/lm.php?url=1000,43,4,20091218142456-GY/Dolamic_Ljiljana_-_When_Stopword_Lists_Make_the_Difference_20091218.pdf
in my opinion, if stopwords annoy your users for very special cases
like 'the the' then, instead consider using commongrams +
defaultsimilarity.discountOverlaps = true so that you still get the
benefits.
as you can see from the above paper, they can be extremely important
depending on the language, they just don't matter so much for
English.
On Tue, Jan 12, 2010 at 9:20 PM, Lance Norskog <goks...@gmail.com>
wrote:
There are a lot of projects that don't use stopwords any more. You
might consider dropping them altogether.
On Mon, Jan 11, 2010 at 2:25 PM, Don Werve <d...@madwombat.com>
wrote:
This is the way I've implemented multilingual search as well.
2010/1/11 Markus Jelsma <mar...@buyways.nl>
Hello,
We have implemented language specific search in Solr using
language
specific fields and field types. For instance, an en_text field
type can
use an English stemmer, and list of stopwords and synonyms. We,
however
did not use specific stopwords, instead we used one list shared
by both
languages.
So you would have a field type like:
<fieldType name="en_text" class="solr.TextField" ...
<analyzer type="">
<filter class="solr.StopFilterFactory" words="stopwords.en.txt">
<filter class="solr.SynonymFilterFactory"
synonyms="synoyms.en.txt">
etc etc.
Cheers,
-
Markus Jelsma Buyways B.V.
Technisch Architect Friesestraatweg 215c
http://www.buyways.nl 9743 AD Groningen
Alg. 050-853 6600 KvK 01074105
Tel. 050-853 6620 Fax. 050-3118124
Mob. 06-5025 8350 In: http://www.linkedin.com/in/markus17
On Mon, 2010-01-11 at 13:45 +0100, Daniel Persson wrote:
Hi Solr users.
I'm trying to set up a site with Solr search integrated. And I
use the
SolJava API to feed the index with search documents. At the
moment I
have only activated search on the English portion of the site.
I'm
interested in using as many features of solr as possible.
Synonyms,
Stopwords and stems all sounds quite interesting and useful but
how do
I set up this in a good way for a multilingual site?
The site don't have a huge text mass so performance issues don't
really bother me but still I'd like to hear your suggestions
before I
try to implement an solution.
Best regards
Daniel
--
Lance Norskog
goks...@gmail.com
--
Robert Muir
rcm...@gmail.com
