Re: thread safe shared IndexSearcher

2007-09-24 Thread Jay Yu
total time to parse a query and run a search. I'll try and get around to posting the code tonight. - Mark Jay Yu wrote: Mark Miller wrote: Good luck Jay. Keep in mind, pretty much all LuceneIndexAccessor does is sync Readers with Writers and allow multiple threads to share the same in

Re: thread safe shared IndexSearcher

2007-09-24 Thread Jay Yu
Mark, Great effort getting the original lucene index accessor package in this shape. I am sure this will benefit a lot of people using Lucene in a multithread env. I have a quick question to ask you: Do you have to use the core Lucene 2.3-dev in order to use the accessor? I will take a look a

Re: thread safe shared IndexSearcher

2007-09-24 Thread Jay Yu
reset analyzer/Dir as in my own version. Jay Mark Miller wrote: One final noteif you are using the IndexAccessor and you are only accessing the index from one JVM, you can use the NoLockFactory and save some sync cost there. Jay Yu wrote: Mark, Great effort getting the original

Re: thread safe shared IndexSearcher

2007-09-25 Thread Jay Yu
code. I will take your comments into account and update the code. I have some other pieces to polish as well. Previously, I extended and built upon the original code, but I can't give it away, so this is my attempt at something lessor, but cleaner. Jay Yu wrote: Thanks for the tip. One

Re: thread safe shared IndexSearcher

2007-09-25 Thread Jay Yu
y, it does work. I am also thinking about how to handle the Analyzer, and I think the solution (the need to init some indexaccessor params) might involve all these issues. - Mark Jay Yu wrote: Mark, Looking at your implementation of the DefaultIndexAccessor regarding the writer, I think t

RangeFilter

2007-07-10 Thread Jay Yu
Hi All, Does anyone know if the RangeFilter is a cached filter? I could not tell from the api. Thanks! Jay - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Re: RangeFilter

2007-07-10 Thread Jay Yu
Thanks for clarifying this, Chris! I agree with you that javadocs usual should doc all they do but often times they skip few important things they do do. Chris Hostetter wrote: : Does anyone know if the RangeFilter is a cached filter? I could not : tell from the api. Generally speaking cla

Re: Search for null

2007-07-23 Thread Jay Yu
If you want performance, a better way might be to assign some special string/value (if it's easy to create) to the missing field of docs and index the field without tokenizing it. Then you may search for that special value to find the docs. Jay Les Fletcher wrote: Does this particular range

Re: Search for null

2007-07-24 Thread Jay Yu
d can cheaply be stored, generated once and used often. Dan On Mon, 2007-07-23 at 13:57 -0700, Jay Yu wrote: If you want performance, a better way might be to assign some special string/value (if it's easy to create) to the missing field of docs and index the field without tokenizing it. Then you

Re: Search for null

2007-07-25 Thread Jay Yu
bits final BitSet filterBitSet = queryFilter.bits(reader); filterBitSet.flip(0,filterBitSet.size()); Now you have a filter that contains document matching the opposite of that specified by the query, and can use in subsequent queries Dan On Tue, 2007-07-24 at 09:40 -0700, Jay Yu wrote: daniel ro

thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
In a multithread app like web app, a shared IndexSearcher could throw a AlreadyClosedException when another thread is trying to update the underlying IndexReader by closing the shared searcher after the index is updated. Searching over the past discussions on this mailing list, I found several

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
k its overlooked far too much. - Mark On 9/19/07, Jay Yu <[EMAIL PROTECTED]> wrote: In a multithread app like web app, a shared IndexSearcher could throw a AlreadyClosedException when another thread is trying to update the underlying IndexReader by closing the shared searcher after t

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
raight from the originally submitted code. 4. I will look into getting together some code that I can share. The multisearcher changes that are need are a couple of one liners really, so at a minimum I will give you the changes needed. - Mark On 9/19/07, Jay Yu <[EMAIL PROTEC

Re: thread safe shared IndexSearcher

2007-09-19 Thread Jay Yu
from the originally submitted code. 4. I will look into getting together some code that I can share. The multisearcher changes that are need are a couple of one liners really, so at a minimum I will give you the changes needed. - Mark On 9/19/07, Jay Yu <[EMAIL PROTECTED]> wrote: Mark,

Re: thread safe shared IndexSearcher

2007-09-20 Thread Jay Yu
less your bulk loading). I've had such a system with 5+ million docs on a standard machine and searches where still well below a second after the first Searcher is cached (and even the first search is darn quick). And that includes a lot of extra crap I am doing. - Mark Jay Yu wrote

Re: thread safe shared IndexSearcher

2007-09-20 Thread Jay Yu
will appreciate it! Thanks for your help! - Mark Jay Yu wrote: Mark, Thanks for sharing your valuable exp. and thoughts. Frankly our system already has most of the functionalities LuceneIndexAcessor offers. The only thing I am looking for is to sync the searchers' close. That's