You should not use the ParallelMultiSearcher. It will not be in the 4.0 release.
On Thu, Mar 17, 2011 at 11:38 AM, Uwe Schindler wrote:
> Hi Ganesh,
>
> this method is also in 2.9.1, it is just inherited from the superclass! You
> have to also look at the complete javadocs. Not every
Sent: Thursday, March 17, 2011 7:09 PM
> To: java-user@lucene.apache.org
> Subject: Re: RE: ParallelMultisearcher
>
> There is no ParallelMultiSearcher.close() in 2.9.1 and its been added in
3.0.
> Thanks for pointing out. I am not aware of.
>
> Regards
> Ganesh
>
&g
There is no ParallelMultiSearcher.close() in 2.9.1 and its been added in 3.0.
Thanks for pointing out. I am not aware of.
Regards
Ganesh
- Original Message -
From: "Uwe Schindler"
To:
Sent: Thursday, March 17, 2011 11:27 PM
Subject: [Bulk] RE: ParallelMultisearcher
&
Hi Ganesh, It's definitely in 3.0.3:
@Override
public void close() throws IOException {
executor.shutdown();
super.close();
}
This code is in the downloaded .src.zip file of Apache Lucene 3.0.3. Are you
sure that you call close() on the ParallelMultiSearcher (e.g. when you are
shing frequently.
Regards
Ganesh
- Original Message -
From: "Devon H. O'Dell"
To:
Sent: Thursday, March 17, 2011 10:35 PM
Subject: Re: ParallelMultisearcher
> 2011/3/17 Ganesh :
>> Is this bug https://issues.apache.org/jira/browse/LUCENE-2249 got fixed in
>
Yes, it's fixed in SVN since June. Lucene 3.0.3 should contain the fix. When
you call ParallelMultiSearcher#close, the Executor is shutdown.
-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
> -Original Message-
> Fr
March 17, 2011 7:03 PM
> Subject: Re: ParallelMultisearcher
>
>
> I tested the same with Lucene 2.9.1 and found very less thread count but with
> 3.0.2 it is very high. Do i need to revert back to Lucene 2.9.1.
>
> Regards
> Ganesh
>
> - Original Message
Is this bug https://issues.apache.org/jira/browse/LUCENE-2249 got fixed in
3.0.3?
Regards
Ganesh
- Original Message -
From: "Ganesh"
To:
Sent: Thursday, March 17, 2011 7:03 PM
Subject: Re: ParallelMultisearcher
I tested the same with Lucene 2.9.1 and found very less th
I tested the same with Lucene 2.9.1 and found very less thread count but with
3.0.2 it is very high. Do i need to revert back to Lucene 2.9.1.
Regards
Ganesh
- Original Message -
From: "Ganesh"
To:
Sent: Thursday, March 17, 2011 5:17 PM
Subject: ParallelMultisearcher
Hello all,
Is there any issue with ParallelMultiSearcher in Lucene 3.0.2? If we search
more frequently then OutOfMemoryError is triggered or It is throwing "Not able
to create native thread".. It is working with 2.9.1.
I generated Heap Dump and found lot of entries for ParallelMul
x, an OutOfMemory error occurs.
Since I need to potentially search across multiple indexes and those indexes
can change from one search query to the next, each user has their own
ParallelMultiSearcher object. Before each search operation, I reconstruct
the ParrallelMultisearcher with the appropriate Se
; I have an index which is 36 GB large. When I perform eight simultaneous
> searches (performed by JMeter) on the index, an OutOfMemory error occurs.
> Since I need to potentially search across multiple indexes and those indexes
> can change from one search query to the next, each user
their own ParallelMultiSearcher object. Before each search
operation, I reconstruct the ParrallelMultisearcher with the appropriate
Searchers to each of the indexes that need to be included for that
particular search query.
The problem is that requiring each user to have their own
ase it makes something different
> than using the core-Lucene ParallelMultiSearcher with RemoteSearchables. Maybe
> on this list somebody knows the answer.
>
>
>
>
>
> On Tue, 4 Aug 2009 07:20:23 -0700 (PDT)
> Otis Gospodnetic wrote:
>
>> Hi Christian,
>>
>
wrong. At least, it would be from interest how Solr
realizes its distributed search, in the case it makes something different
than using the core-Lucene ParallelMultiSearcher with RemoteSearchables. Maybe
on this list somebody knows the answer.
On Tue, 4 Aug 2009 07:20:23 -0700 (PDT)
Otis
om: Christian Reuschling
> To: java-user@lucene.apache.org
> Sent: Tuesday, August 4, 2009 5:50:16 AM
> Subject: ParallelMultiSearcher and idf
>
> Hello,
>
> when searching over multiple indices, we create one IndexReader for each
> index,
> and wrap them into a MultiReader
ering whether there is the possibility to get the advantages of both
scenarios, e.g. by first summing up the query terms-related document
frequencies, and sending them together with the query to every (remote)searcher
of ParallelMultiSearcher, for scoring.
Maybe this is exactly what ParallelMultiSearcher
I want to sort results of a query according to a specific field(date).I have
a parallelMultiSearcher with some underlying remote searchers. Indexes in
remote searchables are large and enabling sort will make searching slow. As
I understood from previous threads, a solution is: searching without
On Wed, Oct 04, 2006 at 01:55:06PM +, eks dev said:
> have you considered hadoop "light" mesagging RPC, should have
> significantly smaller latencies than RMI
Yes, it's one of the things I'm looking at.
-
To unsubscribe, e-
on big index by using ParallelMultiSearcher is
slow...
On Wed, Oct 04, 2006 at 08:14:38AM -0400, Haines, Ronald C. (LNG-DAY) said:
> I too am interested in learning more about a large scale distributed
> Lucene model.
I'm also building a large scale (billions of documents
> Prelimary experimentation with a RemoteSearch/ParallelMultiSearcher
> combination found that there were issues with the RMI causing
> significant blocking.
>
> I'm currently playing around with trying alternative messaging
> approaches so that I can also load balance req
emoteSearch/ParallelMultiSearcher
combination found that there were issues with the RMI causing
significant blocking.
I'm currently playing around with trying alternative messaging
approaches so that I can also load balance requests.
One of the things I haven't decided on is the optimum size for t
bject remotely and SearchMaster collects
top N of Hit from Hits then SearchMaster sort it.
I tested ParallelMultiSearcher performance.
it makes and starts thread serially.
Then wait for all threads ended.
But it is threaded, so searching is parallelly on remote server,
I insert debug program that
ver with splitted index (divided by 10) is about 50 ms.
And on ParallelMultiSearcher with 10 of remote searchable,
each RemoteSearchable returns in about 50 - 100 ms,
and ParallelMultiSearcher returns also 50 - 100 ms, because of
threading.
but Hits Searcher.search(Query, Sort) responds in about 500
There will be (# of terms * # of remotes) of these
remote calls made for each of the above methods.
And, I think the ParallelMultiSearcher will make all of these calls
serially before it starts to thread the search process. I have found
that this, serially, can account for quite a bit of t
0 ms.
On one server with splitted index (divided by 10) is about 50 ms.
And on ParallelMultiSearcher with 10 of remote searchable,
each RemoteSearchable returns in about 50 - 100 ms,
and ParallelMultiSearcher returns also 50 - 100 ms, because of
threading.
but Hits Searcher.search(Q
d by 10) is about 50 ms.
And on ParallelMultiSearcher with 10 of remote searchable,
each RemoteSearchable returns in about 50 - 100 ms,
and ParallelMultiSearcher returns also 50 - 100 ms, because of
threading.
but Hits Searcher.search(Query, Sort) responds in about 500 - 1000 ms.
I think that Search
inding
out what happens when you forget all the ParallelMultiSearcher stuff, all
the RMI stuff etc, and just see what your performance is on one of your
index parts locally. Once that is answered, extend to RMI, then the
Parallel, at each step seeing if your performance degrades unacceptably.
That&#
Hi,
I have a question about ParallelMultiSearcher performance.
I want to search documents on about 10 gigabytes of index.
(The index has 10,000,000 documents.)
I get very slow performance using IndexSearcher with ONE index normally.
Then I tried to use ParallelMultiSearcher with 10 servers of
Hello, Yonik.
>> ok. I have big issue when I try to search ParallelMultiSearcher for
>> PrefixQuery. This query is being rewritten to BooleanQuery during
>> search. This causes Similarity to calculate docFreq for each Term in the
>> BooleanQuery. So if we have a
On 9/21/06, Yura Smolsky <[EMAIL PROTECTED]> wrote:
ok. I have big issue when I try to search ParallelMultiSearcher for
PrefixQuery. This query is being rewritten to BooleanQuery during
search. This causes Similarity to calculate docFreq for each Term in the
BooleanQuery. So if we have a
Hello, Ronnie.
RK> Dont ask to ask, just ask! ;)
ok. I have big issue when I try to search ParallelMultiSearcher for
PrefixQuery. This query is being rewritten to BooleanQuery during
search. This causes Similarity to calculate docFreq for each Term in the
BooleanQuery. So if we have a lot
Dont ask to ask, just ask! ;)
Citerar Yura Smolsky <[EMAIL PROTECTED]>:
> Hello, java-user.
>
> Does anyone here uses ParallelMultiSearcher for searching big arrays
> of data? I have some questions about PrefixQuery search..
>
> Thanks in advance.
>
Hello, java-user.
Does anyone here uses ParallelMultiSearcher for searching big arrays
of data? I have some questions about PrefixQuery search..
Thanks in advance.
--
Yura Smolsky,
http://altervisionmedia.com/
-
To
an avoid the 'call
HRCLD> per Term' per remote and just make a single docFreq call per remote.
HRCLD> You might have to extend the ParallelMultiSearcher and create a threaded
HRCLD> docFreq method.
--
Yura Smolsky,
http://altervisionmedia.com/
-
emote.
You might have to extend the ParallelMultiSearcher and create a threaded
docFreq method.
-Original Message-
From: Yura Smolsky [mailto:[EMAIL PROTECTED]
Sent: Friday, September 15, 2006 8:54 AM
To: java-user@lucene.apache.org
Subject: Re: ParallelMultiSearcher and docFreq
Hello, Yura.
Does anyone understand my email? Maybe my English is too bad...
Thanks.
YS> Here is the situation. I have ParallelMultiSearcher object
YS> initializated with two or more RemoteSearchable's.
YS> I run PrefixQuery search on some keyword field, say "link"
Hello.
Here is the situation. I have ParallelMultiSearcher object
initializated with two or more RemoteSearchable's.
I run PrefixQuery search on some keyword field, say "link". When I run
search starting just with letter "w" (link:w*) then I should have like 5k
results.
nie Kolehmainen [mailto:[EMAIL PROTECTED]
Sent: Tuesday, December 20, 2005 11:59 PM
To: java-user@lucene.apache.org
Subject: Re: NullPointerException in ParallelMultiSearcher
I believe the most common case is when you do a search with Sort (AUTO
feature), and one of the fields you sort by is empty.
ect: NullPointerException in ParallelMultiSearcher
Hi,
We are using lucene v1.4.3 for some time, in general it is working well.
We often try to search multiple collections at the same time, so we
are using ParallelMultiSearcher, but sometimes we got the following
exception:
java.
Hi,
I have not received any feedback yet, any comments
would be greatly appreciated!
Lisheng
-Original Message-
From: Zhang, Lisheng
Sent: Thursday, December 01, 2005 12:30 PM
To: 'java-user@lucene.apache.org'
Subject: NullPointerException in ParallelMultiSearcher
Hi,
We
Hi,
We are using lucene v1.4.3 for some time, in general it is working well.
We often try to search multiple collections at the same time, so we
are using ParallelMultiSearcher, but sometimes we got the following
exception:
java.lang.NullPointerException
at
On Sep 18, 2005, at 7:52 PM, Philip Peterson wrote:
Is there a way to, at run time, boost one index over another in a
ParallelMultiSearcher.
So if I have 4 indexes a,b,c,d and user A searches their results lean
toward index a
and user B searches his results are boosted toward index b etc
Hello,
Is there a way to, at run time, boost one index over another in a
ParallelMultiSearcher.
So if I have 4 indexes a,b,c,d and user A searches their results lean
toward index a
and user B searches his results are boosted toward index b etc.
Thanks so much in advance,
I tried to see if
44 matches
Mail list logo
