Re: add CJKTokenizer to solr

2007-06-26 Thread Daniel Alheiros
Hi Hoss. Yes, it's the tricky part when re-structuring configs... One possible solution is, when you create a new schema, you offer a conversion tool... Other is to define a version on the config and depending on the version, the expected structure will be different. I'm sure you know this all,

Re: add CJKTokenizer to solr

2007-06-26 Thread Chris Hostetter
: conversion tool... Other is to define a version on the config and : depending on the version, the expected structure will be different. FYI: schema.xml does have this ... it's one of hte ,schema attributes. we've only ever reved it once when multiValue fields were added because we wanted to

Re: add CJKTokenizer to solr

2007-06-25 Thread Chris Hostetter
: filter factory=solr.ReflectionFilterFactory class=...CJKTokenFilter / : : I think this way, the config terms are a bit clearer... What do you think? in general, do i think it would be better if the filter and tokenizer declarations used factory as the attribute instead of class? ...yes. So i

RE: add CJKTokenizer to solr

2007-06-22 Thread Xuesong Luo
To: solr-user@lucene.apache.org Subject: Re: add CJKTokenizer to solr I'm sorry. Because it was not possible to append it, it sends it again. I got the error below after adding CJKTokenizer to schema.xml. I checked the constructor of CJKTokenizer, it requires a Reader parameter, I guess that's

Re: add CJKTokenizer to solr

2007-06-22 Thread Daniel Alheiros
Hi Hoss. I've done a few tests using reflection to instantiate a simple object and the results will vary a lot depending on the JVM. As the JVM optimizes code as it is executed it will vary depending on the usage, but I think we have something to consider: If done 1,000 samples (5 clean X loop

Re: add CJKTokenizer to solr

2007-06-22 Thread Daniel Alheiros
Sorry I've confused things a bit... The thread safeness have to be considered only on the Tokenizers, not on the factories. So are the Tokenizers thread safe? Regards, Daniel On 22/6/07 11:36, Daniel Alheiros [EMAIL PROTECTED] wrote: Hi Hoss. I've done a few tests using reflection to

Re: add CJKTokenizer to solr

2007-06-22 Thread Otis Gospodnetic
: Friday, June 22, 2007 12:43:50 PM Subject: Re: add CJKTokenizer to solr Sorry I've confused things a bit... The thread safeness have to be considered only on the Tokenizers, not on the factories. So are the Tokenizers thread safe? Regards, Daniel On 22/6/07 11:36, Daniel Alheiros [EMAIL

Re: add CJKTokenizer to solr

2007-06-22 Thread Chris Hostetter
: Sorry I've confused things a bit... The thread safeness have to be : considered only on the Tokenizers, not on the factories. So are the : Tokenizers thread safe? nope ... they are constructed using Readers and mainting state about the text they are processing ... the only api is a next()

Re: add CJKTokenizer to solr

2007-06-22 Thread Mike Klaas
On 21-Jun-07, at 10:22 PM, Chris Hostetter wrote: like i said though: i'm in favore of factories like this ... i just don't think we should do anything to hide their use and make refering to Tokenizer or TOkenFilter class names directly use reflection magicly. What would be the best way

Re: add CJKTokenizer to solr

2007-06-18 Thread Toru Matsuzawa
I got the error below after adding CJKTokenizer to schema.xml. I checked the constructor of CJKTokenizer, it requires a Reader parameter, I guess that's why I get this error, I searched the email archive, it seems working for other users. Does anyone know what is the problem?

Re: add CJKTokenizer to solr

2007-06-18 Thread Toru Matsuzawa
I'm sorry. Because it was not possible to append it, it sends it again. I got the error below after adding CJKTokenizer to schema.xml. I checked the constructor of CJKTokenizer, it requires a Reader parameter, I guess that's why I get this error, I searched the email archive, it seems

Re: add CJKTokenizer to solr

2007-01-30 Thread zha jimmy
Thank you all, it's works now:). 2007/1/30, James liu [EMAIL PROTECTED]: he now is ok. -- regards jl

Re: add CJKTokenizer to solr

2007-01-29 Thread Erik Hatcher
On Jan 29, 2007, at 1:08 AM, zha jimmy wrote: hi, all I am try to config solr to support chinese tokenize。 I saw the tips in schema.xml: !-- One can also specify an existing Analyzer class that has a default constructor via the class attribute on the analyzer element

Re: add CJKTokenizer to solr

2007-01-29 Thread Chris Hostetter
: I realized that solr do not have the CJK package ,but how can I : add it : in? : : You need to add the analyzers JAR from Lucene's contrib area to your : Solr application, under WEB-INF/lib. You can get that JAR from the : latest Lucene release distribution. it's acctually eazier then

Re: add CJKTokenizer to solr

2007-01-29 Thread James liu
he now is ok. -- regards jl