Re: Using multiple language stop words in Solr Core

[Markus Jelsma]

Hell Abhay,

Do not enable stopwords unless you absolutely know what you are doing. In
general, it is a bad practice that somehow still lingers on.

But to answer the question, you must have one field and fieldType for each
language, so language specific filters go there. Also, using edismax and
multi-language search using mm (minimum should match) with stopwords
enabled spells trouble.

Set up per language fieldTypes without stopwords.

Regards,
Markus

Op do 11 feb. 2021 om 12:44 schreef Abhay Kumar <
abhay.ku...@anjusoftware.com>:

> Hello Team,
>
>
>
> Solr provides some data type out of box in managed schema for different
> languages such as english, french, japanies etc.
>
>
>
> We are using common data type "text_general" for fields declaration and
> using stopwards.txt for stopword filtering.
>
>
>
>  autoGeneratePhraseQueries="true" positionIncrementGap="100"
> multiValued="true">
>
> 
>
>   
>
>ignoreCase="true"/>
>
>   
>
>minGramSize="1"/>
>
> 
>
> 
>
>   
>
>ignoreCase="true"/>
>
>ignoreCase="true" synonyms="synonyms.txt"/>
>
>   
>
> 
>
>   
>
>
>
> While syncing data to Solr core we are importing different languages text
> in the fields such as french, english, german etc.
>
>
>
> My query is shall we use all different language stopwords into same
> "stopwards.txt" file or how solr use different language stopwords?
>
>
>
>
>
>
>
> *Warm Regards,*
>
>
>
> *Abhay Kumar* | Lead Developer
>
> 401/402, Pride Portal, Shivaji Housing Society, Off. S. B. Road | Shivaji
> Nagar, Pune-411 016
> +91 20 2563 1011 | Mobile: +91 9096644108
> anjusoftware.com
>
> 
> 
> 
> 
>
>
>
>
>
>
>
> *Confidentiality Notice  This email message, including
> any attachments, is for the sole use of the intended recipient and may
> contain confidential and privileged information. Any unauthorized view,
> use, disclosure or distribution is prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message. Anju Software, Inc. 4500 S. Lakeshore Drive, Suite
> 620, Tempe, AZ USA 85282.*
>

Using multiple language stop words in Solr Core

[Abhay Kumar]

Hello Team,

Solr provides some data type out of box in managed schema for different 
languages such as english, french, japanies etc.

We are using common data type "text_general" for fields declaration and using 
stopwards.txt for stopword filtering.



  
  
  
  


  
  
  
  

  

While syncing data to Solr core we are importing different languages text in 
the fields such as french, english, german etc.

My query is shall we use all different language stopwords into same 
"stopwards.txt" file or how solr use different language stopwords?



Warm Regards,

Abhay Kumar | Lead Developer
401/402, Pride Portal, Shivaji Housing Society, Off. S. B. Road | Shivaji 
Nagar, Pune-411 016
+91 20 2563 1011 | Mobile: +91 9096644108
anjusoftware.com
[cid:image001.png@01D70099.4ACD8C20][cid:image002.png@01D70099.4ACD8C20][cid:image003.png@01D70099.4ACD8C20][cid:image004.png@01D70099.4ACD8C20]



Confidentiality Notice

This email message, including any attachments, is for the sole use of the 
intended recipient and may contain confidential and privileged information. Any 
unauthorized view, use, disclosure or distribution is prohibited. If you are 
not the intended recipient, please contact the sender by reply email and 
destroy all copies of the original message. Anju Software, Inc. 4500 S. 
Lakeshore Drive, Suite 620, Tempe, AZ USA 85282.