Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

: We found if we were using 2 IndexSearcher, we would get 10% performance : benefit. : But if we increased the number of IndexSearcher from 2, the performance : improvement became slight even worse. Why use more then 2 IndexSearchers? Typically 1 is all you need, except for when you want to

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

Please trace the codes into the Lucene when searching. Here is a table about how invokations are called. The trace log: *Steps* *ClassName* *Functions* *Description* 1. org.apache.lucene.search.Searcher public final Hits search(Query query) It will call another search function. 2. org.apac

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

ooops, the table seems twisted. Can you see that clearly? On 5/9/06, yueyu lin <[EMAIL PROTECTED]> wrote: Please trace the codes into the Lucene when searching. Here is a table about how invokations are called. The trace log: *Steps* *ClassName* *Functions* *Description* 1. org.apache.l

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

One IndexSearcher is one IndexSearcher instance. The instance has a lot of functions. Unfortunately they will call another synchronized function in other class's instance (TermInfosReader). That's the point why we need two IndexSearchers. But two searchers will cost double cache memory. It's not w

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

che.org Sent: Tuesday, May 9, 2006 3:53:55 AM Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Please trace the codes into the Lucene when searching. Here is a table about how invokations are called. The trace log: *Steps* *Cla

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

The best search performance is achieved using a single IndexSearcher shared by multiple threads. Peter Keegan has demonstrated rates of up to 400 searches per second on eight-CPU machines using this approach: http://www.mail-archive.com/java-user@lucene.apache.org/msg05074.html So the synchro

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

ene.apache.org Sent: Tuesday, May 9, 2006 3:53:55 AM Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Please trace the codes into the Lucene when searching. Here is a table about how invokations are called. The trace log:

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

My assumption is that every query is relatively quick. If the times lapsed in other process when querying, the ensureIndexIsRead() function will not cause a lot of problems. If not, the ensureIndexIsRead() function will be a bottle neck. I could understand that a lot of systems' queries are quiet

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

-- From: yueyu lin <[EMAIL PROTECTED]> To: java-dev@lucene.apache.org Sent: Tuesday, May 9, 2006 3:53:55 AM Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Please trace the codes into the Lucene when searching. Here is a table about how invok

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

apache.org Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. My assumption is that every query is relatively quick. If the times lapsed in other process when querying, the ensureIndexIsRead() function will not cause a lot of problems. I

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

, May 09, 2006 8:46 PM To: java-dev@lucene.apache.org; Otis Gospodnetic Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Oh,please believe in me that I've forced the JVM to print the thread dump. It waited here indeed. I'll try to post

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

y, May 09, 2006 8:46 PM To: java-dev@lucene.apache.org; Otis Gospodnetic Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Oh,please believe in me that I've forced the JVM to print the thread dump. It waited here indeed. I'll try to post the patch

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

Yueyu Lin, Your patch below looks suspiciously like the double-checked locking anti-pattern, and is not guaranteed to work. There really isn't a way to safely lazily initialize without using synchronized or volatile. -Yonik http://incubator.apache.org/solr Solr, the open-source Lucene search ser

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

In java, call a synchronized function in a synchronized block, if they have the same mutex object, nothing will happen. If they have different mutex objects, something may be screwed up. On 5/10/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: Yueyu Lin, Your patch below looks suspiciously like the

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. Yueyu Lin, Your patch below looks suspiciously like the double-checked locking anti-pattern, and is not guaranteed to work. There really isn't a way to safely lazily initialize without using synchroniz

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

Here is a reference to double-checked locking. Many people have tried to get around synchronization during lazy initialization - AFAIK, none have succeeded. With the new memory model in Java5, you can get away with just volatile, which is like half a synchronization (a read barrier + a write bar

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

On 5/9/06, Robert Engels <[EMAIL PROTECTED]> wrote: I am fairly certain his code is ok, since it rechecks the initialized state in the synchronized block before initializing. That "recheck" is why the pattern (or anti-pattern) is called double-checked locking :-) -Yonik http://incubator.apache

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

: > I am fairly certain his code is ok, since it rechecks the initialized state : > in the synchronized block before initializing. : : That "recheck" is why the pattern (or anti-pattern) is called : double-checked locking :-) More specificly, this is functionally half way between example labeled

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

ration. > > -Original Message- > From: yueyu lin [mailto:[EMAIL PROTECTED] > Sent: Tuesday, May 09, 2006 8:46 PM > To: java-dev@lucene.apache.org; Otis Gospodnetic > Subject: Re: Multiple threads searching in Lucene and the synchronized > issue. -- solution attached. > >

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

I met these problem before indeed.The compiler did something optimized for me that was bad for me when I see the byte-codes. When I'm using a function local variable, m_indexTerms and in JDK1.5.06, it seems ok. Whether it will break in other environments, I still don't know about it. On 5/10/06, Y

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

, May 09, 2006 11:32 PM To: java-dev@lucene.apache.org Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. I met these problem before indeed.The compiler did something optimized for me that was bad for me when I see the byte-codes. When I'm us

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

ailto:[EMAIL PROTECTED] Sent: Tuesday, May 09, 2006 11:32 PM To: java-dev@lucene.apache.org Subject: Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. I met these problem before indeed.The compiler did something optimized for me that was bad for me when I se

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

: I think you could use a volatile primitive boolean to control whether or not : the index needs to be read, and also mark the index data volatile and it : SHOULD PROBABLY work. : : But as stated, I don't think the performance difference is worth it. My understanding is: 1) volatile will only h

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

I understand that. Thanks for all. I will still use the original Lucene jar and will continue to dig Lucene. Wish I would find something useful for all of you. :) On 5/10/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: : I think you could use a volatile primitive boolean to control whether or

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

10, 2006 12:51 AM : To: Lucene Dev : Subject: RE: Multiple threads searching in Lucene and the synchronized : issue. -- solution attached. : : : : : I think you could use a volatile primitive boolean to control whether or : not : : the index needs to be read, and also mark the index data volatile an

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

nding). -Original Message- From: Chris Hostetter [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 10, 2006 12:51 AM To: Lucene Dev Subject: RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached. : I think you could use a volatile primitive boolean to control wheth

Re: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

On 5/10/06, Robert Engels <[EMAIL PROTECTED]> wrote: I think you could use a volatile primitive boolean to control whether or not the index needs to be read, and also mark the index data volatile and it SHOULD PROBABLY work. No, that still doesn't work. volatile doesn't quite mean what you thin

RE: Multiple threads searching in Lucene and the synchronized issue. -- solution attached.

that JVMs will be able to implement synchronized more efficiently. -Original Message- From: Yonik Seeley [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 10, 2006 10:07 AM To: java-dev@lucene.apache.org; [EMAIL PROTECTED] Subject: Re: Multiple threads searching in Lucene and the