Re: IndexSearcher.doc thread safe problem
This problem is solved.
Queue is OK in multi-thread.
Lucene searcher is OK in multi-thread.
Hoss as saying the problem was found while demonstrate testing.
My method had the problem.
Thanks to helpers.
2014-07-10 11:05 GMT+09:00 Chris Hostetter :
>
> : 4. Syncronized searcher.doc method call in multi-thread(like this: public
> : synchronized Document getValue( IndexSearcher searcher, int docId ) {
> : return searcher.doc( docId ); })
> : ==> every execution is same.
> :but If I use this method, It is no difference with single
> thread
> : performance.
> :
> : What do you think about it?
>
> You're asking us about the behavior of IndexSearcher.doc from
> multi-threaded code, but you haven't shown us enough code to even guess as
> to what your problem might be -- let alone reproduce it.
>
> can you please submit a cully self contained program -- that builds an
> index and then searches it -- and which demonstrates the problem you are
> having? That way folks trying to help you will have actual code they can
> run on their machine to understand the problem you are describing.
>
>
>
> -Hoss
> http://www.lucidworks.com/
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
Re: IndexSearcher.doc thread safe problem
: 4. Syncronized searcher.doc method call in multi-thread(like this: public
: synchronized Document getValue( IndexSearcher searcher, int docId ) {
: return searcher.doc( docId ); })
: ==> every execution is same.
:but If I use this method, It is no difference with single thread
: performance.
:
: What do you think about it?
You're asking us about the behavior of IndexSearcher.doc from
multi-threaded code, but you haven't shown us enough code to even guess as
to what your problem might be -- let alone reproduce it.
can you please submit a cully self contained program -- that builds an
index and then searches it -- and which demonstrates the problem you are
having? That way folks trying to help you will have actual code they can
run on their machine to understand the problem you are describing.
-Hoss
http://www.lucidworks.com/
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
Re: IndexSearcher.doc thread safe problem
Thanks answer Ian
I more check this problem..(Lucene 4.8.1)
1. acceptsDocsOutOfOrder() return false
==> ineffective
2. acceptsDocsOutOfOrder() return true
==> ineffective
3. ConncurrentLinkedQueue add() and poll() methods check value in
multi-thread
==> the queue is no problem.
4. Syncronized searcher.doc method call in multi-thread(like this: public
synchronized Document getValue( IndexSearcher searcher, int docId ) {
return searcher.doc( docId ); })
==> every execution is same.
but If I use this method, It is no difference with single thread
performance.
What do you think about it?
Thanks
2014-07-10 2:04 GMT+09:00 Ian Lea :
> It's more likely to be a demonstration that concurrent programming is
> hard, results often hard to predict and debugging very hard.
>
> Or perhaps you simply need to add acceptsDocsOutOfOrder() to your
> collector, returning false.
>
> Either way, hard to see any evidence of a thread-safety problem in lucene.
>
> If adding acceptsDocsOutOfOrder() doesn't fix it, I suggest you verify
> that your queue is getting the values you expect, in the order you
> expect, consistently. Then worry about the display part, first
> checking everything without any lucene calls.
>
>
> --
> Ian.
>
> On Wed, Jul 9, 2014 at 5:59 AM, 김선무 wrote:
> > Hi all,
> >
> > I know IndexSearcher is thread safe.
> > But IndexSearcher.doc is not thread safe maybe...
> >
> > I try to below
> >
> > First, I extract docID at index directory. And that docID add on
> > queue(ConcurrentLinkedQueue)
> >
> > Second, extract field value using docID poll at this queue after extract
> > process end. This process is work to multi-threads.
> >
> > For this I used the following summation code below:
> > searcher.search( query, filter, new Collector() { public void collect(
> int
> > doc ) { queue.add( docBase + doc ) } );
> > Thread thread1 = new Thread( () -> { while( !queue.isEmpty() ) {
> > System.out.println( searcher.doc(queue.poll()).get("content") ); } } );
> > Thread thread2 = new Thread( thread1 );
> > thread1.start();
> > thread2.start();
> > ---
> >
> > Result was different in every execution.
> >
> > My method is wrong? or IndexSearcher bug?
> >
> > Please help me
>
> -
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
>
>
Re: IndexSearcher.doc thread safe problem
It's more likely to be a demonstration that concurrent programming is
hard, results often hard to predict and debugging very hard.
Or perhaps you simply need to add acceptsDocsOutOfOrder() to your
collector, returning false.
Either way, hard to see any evidence of a thread-safety problem in lucene.
If adding acceptsDocsOutOfOrder() doesn't fix it, I suggest you verify
that your queue is getting the values you expect, in the order you
expect, consistently. Then worry about the display part, first
checking everything without any lucene calls.
--
Ian.
On Wed, Jul 9, 2014 at 5:59 AM, 김선무 wrote:
> Hi all,
>
> I know IndexSearcher is thread safe.
> But IndexSearcher.doc is not thread safe maybe...
>
> I try to below
>
> First, I extract docID at index directory. And that docID add on
> queue(ConcurrentLinkedQueue)
>
> Second, extract field value using docID poll at this queue after extract
> process end. This process is work to multi-threads.
>
> For this I used the following summation code below:
> searcher.search( query, filter, new Collector() { public void collect( int
> doc ) { queue.add( docBase + doc ) } );
> Thread thread1 = new Thread( () -> { while( !queue.isEmpty() ) {
> System.out.println( searcher.doc(queue.poll()).get("content") ); } } );
> Thread thread2 = new Thread( thread1 );
> thread1.start();
> thread2.start();
> ---
>
> Result was different in every execution.
>
> My method is wrong? or IndexSearcher bug?
>
> Please help me
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
IndexSearcher.doc thread safe problem
Hi all,
I know IndexSearcher is thread safe.
But IndexSearcher.doc is not thread safe maybe...
I try to below
First, I extract docID at index directory. And that docID add on
queue(ConcurrentLinkedQueue)
Second, extract field value using docID poll at this queue after extract
process end. This process is work to multi-threads.
For this I used the following summation code below:
searcher.search( query, filter, new Collector() { public void collect( int
doc ) { queue.add( docBase + doc ) } );
Thread thread1 = new Thread( () -> { while( !queue.isEmpty() ) {
System.out.println( searcher.doc(queue.poll()).get("content") ); } } );
Thread thread2 = new Thread( thread1 );
thread1.start();
thread2.start();
---
Result was different in every execution.
My method is wrong? or IndexSearcher bug?
Please help me
