It should work, but if you want to handle multiple languages in ONE index you end up with a lot of filters and fields handled with different analyzers in a SINGLE configuration.
On Wed, Oct 15, 2008 at 3:03 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > > But about stopwords and stemming, is it a real issue if on one core I've > several stemming and stopwords(with a different name), it should work? > > > > Hannes Carl Meyer-2 wrote: > > > > Hi, > > > > yes, if you don't handle (stopwords, stemming etc.) a specific language > > you > > should create a general core. > > > > In my project I'm supporting 10 languages and if I get unsupported > > languages > > it is going to be logged and discarded right away! > > > > Boosting on multiple cores is indeed a problem. An idea would be to merge > > the result sets from core0 and core1 and sort by scoring? > > > > Regards > > > > Hannes > > > > On Wed, Oct 15, 2008 at 1:50 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > > > >> > >> > >> ok MultiCore is handy indeed to don't have this big index wich manage > >> every > >> language, > >> but when you have one modification to do you have to do it on all of > >> them. > >> > >> And the point as well is it's complicate too boost more one language > than > >> another one, > >> ie with an Italian search video, if we don't have that much video then > it > >> might be more interesting to bring back english one. > >> > >> And if there is some language like Slovakia which are not managed by the > >> website but people can come from there ... so the video will be stored > in > >> core0 which will be all language which are not english, spanish, germany > >> .. > >> french. > >> so this kind of garbage core for every language which are not managed > ... > >> and I think it might be hard to manage. > >> > >> What do you think? > >> > >> > >> > >> Hannes Carl Meyer-2 wrote: > >> > > >> > I attached an example for you. > >> > > >> > The challenge with MultiCore is on the client's search logic. It would > >> > help > >> > if you know which language the person wants to search through. If not > >> you > >> > would have to perform multiple requests to the multiple cores. > Ordinary > >> > logic would be: > >> > > >> > 1. search "chien" in core0 (english) > >> > 2. if #1 returned zero results search for "chien" in core1 (french) > >> > > >> > --- > >> > > >> > In your client you could even parallelize the requests to minimize > >> waiting > >> > time. > >> > > >> > *One feature I didn't try yet is the DistributedSearch (and how it > will > >> > help > >> > with multiple cores)*, find it here: > >> > http://wiki.apache.org/solr/DistributedSearch > >> > > >> > Regards, > >> > > >> > Hannes > >> > > >> > On Tue, Oct 14, 2008 at 4:26 PM, sunnyfr <[EMAIL PROTECTED]> > wrote: > >> > > >> >> > >> >> Thanks for this explanation, but just to get it properly : > >> >> > >> >> One core per language, so with the same field and schema just the > >> >> language > >> >> part and management which is different? > >> >> and one core which consider every language which are not managed by > >> solr > >> >> like russian or ??? > >> >> so different request to the dabase.... > >> >> ok > >> >> > >> >> Just don't get really when you look for the word 'chien' on the > >> english > >> >> website I want get back result from french video because chien is > >> french > >> >> so > >> >> if it doesn't find any english video with chien I need my french > video > >> >> then. > >> >> > >> >> Exactly the same for user's core, if somebody look for 'chien' and > >> there > >> >> is > >> >> one user with exactly the same username I would like to show it up. > >> >> > >> >> thanks for your time, really, > >> >> > >> >> > >> >> > >> >> John E. McBride wrote: > >> >> > > >> >> > Fairly nebulous requirements, but I recently was involved in a > >> >> > multilingual search platform. > >> >> > > >> >> > The approach, translated to solr 1.3 would be to use multicore - > one > >> >> > core per geography. Then a schema.xml per core, each with a > >> different > >> >> > language in the porter algorithm, stopwords etc - taken from > >> snowball. > >> >> > > >> >> > Then on the german front end you make requests to the de core, on > >> the > >> >> > english front end make requests to the english core. > >> >> > > >> >> > This is much simpler than sorting every language in the one index, > >> for > >> >> > example german queries will need to be run through the german query > >> >> > filters etc. If you have all languages in one schema, then you > will > >> >> > have to do some front end logic to map the query to the correct > >> field. > >> >> > > >> >> > You have failed to consider internationalisation of the query side > >> of > >> >> > the process - your field type merely have analysis filters. > >> >> > > >> >> > Additionally, if the data source for each different geography is > >> >> > different it makes sense to separate the indexes and subsequently > >> the > >> >> > ingestion mechanisms and schedules. > >> >> > > >> >> > Just a few thoughts. > >> >> > > >> >> > John > >> >> > > >> >> > sunnyfr wrote: > >> >> >> Hi, > >> >> >> > >> >> >> I would like to manage properly multi language search motor, > >> >> >> I would like your advice about what have I done. > >> >> >> > >> >> >> Solr1.3 > >> >> >> tomcat55 > >> >> >> > >> >> >> http://www.nabble.com/file/p19954805/schema.xml schema.xml > >> >> >> > >> >> >> Thanks a lot, > >> >> >> > >> >> >> > >> >> > > >> >> > > >> >> > > >> >> > >> >> -- > >> >> View this message in context: > >> >> > >> > http://www.nabble.com/Multi-language-solr1.3-what-would-you-reckon--tp19954805p19974618.html > >> >> Sent from the Solr - User mailing list archive at Nabble.com. > >> >> > >> >> > >> > > >> > Solr1.3 MultiCore Scenario > >> > > >> > core0 (french) core1 (english) ... > >> core8 (russian) > >> > |schema.xml schema.xml > >> schema.xml > >> > |- analyzers |- analyzers |- > >> analyzers > >> > |-- FrenchAnalyzer |-- EnglishAnalyzer |-- > >> RussianAnalyzer > >> > |-- FrenchStops |-- EnglishStops > >> |-- > >> RussianStops > >> > |- fields |- fields > >> |- fields > >> > |-- title |-- title > >> |-- title > >> > |-- description |-- description > >> |-- > >> description > >> > |-- id |-- id > >> |-- id > >> > > >> > >> -- > >> View this message in context: > >> > http://www.nabble.com/Multi-language-solr1.3-what-would-you-reckon--tp19954805p19991949.html > >> Sent from the Solr - User mailing list archive at Nabble.com. > >> > >> > > > > > > -- > View this message in context: > http://www.nabble.com/Multi-language-solr1.3-what-would-you-reckon--tp19954805p19993036.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
