Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Cheng
Will do thanks On Wed, Jan 11, 2012 at 3:37 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > Yes, it's best to share one IndexSearcher/IndexReader across all > threads... and if you ever find evidence this hurts concurrency then > please post back :) > > Mike McCandless > > http://blo

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Michael McCandless
Yes, it's best to share one IndexSearcher/IndexReader across all threads... and if you ever find evidence this hurts concurrency then please post back :) Mike McCandless http://blog.mikemccandless.com On Wed, Jan 11, 2012 at 3:29 PM, Cheng wrote: > Will do if I see a perf gain. > > The other is

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Cheng
Will do if I see a perf gain. The other issue is that in each thread my apps will not only do indexing but searching. That means I will have to pass through the ram directory instance, along with the writer instance, to every thread so that the searcher can be built on. Should I create a same rea

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Michael McCandless
Yes that would work fine but you should see a net perf loss by doing so (once you include time to flush/sync the RAMDir to an FSDir). If you see a perf gain then please report back! Mike McCandless http://blog.mikemccandless.com On Wed, Jan 11, 2012 at 3:09 PM, Cheng wrote: > Can I create

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Cheng
Can I create a RAMDirectory based writer and have it work cross all threads? In the sense, I would like to use RAMDirectory every where and have the RAMDirectory written to FSDirectory in the end. I suppose that should work, right? On Wed, Jan 11, 2012 at 2:31 PM, Michael McCandless < luc...@mik

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Michael McCandless
On Wed, Jan 11, 2012 at 1:32 PM, dyzc2010 wrote: > Mike, do you mean if I create a FSDirectory based writer in first place, then > the writer should be used in every thread rather than create a new > RAMDirectory based writer in that thread? Right. > What about I do want to use RAMDirectory t

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread dyzc2010
? -- Original -- From: "Michael McCandless"; Date: Thu, Jan 12, 2012 02:11 AM To: "java-user"; Subject: Re: Seem contradictive -- indexwriter in handling multiple threads You shouldn't have to write first to intermediate RAMDirectorys anymore just s

Re: Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Michael McCandless
You shouldn't have to write first to intermediate RAMDirectorys anymore just share a single IndexWriter instance across all of your threads. Mike McCandless http://blog.mikemccandless.com On Wed, Jan 11, 2012 at 12:19 PM, Cheng wrote: > I have read a lot about IndexWriter and multi-threadin

Seem contradictive -- indexwriter in handling multiple threads

2012-01-11 Thread Cheng
I have read a lot about IndexWriter and multi-threading over the Internet. It seems to me that the normal practice is: 1) use a same indexwriter instance for multiple threads; 2) create an individual RAMDirectory per threads; 3) use addIndexes(Directory[]) methods to add to a local drive folder al