Sorry I missed it in the solrconfig.xml (my bad). I wasn't looking for
it in the right place.
Thijs
On 27-5-2010 6:41, Chris Hostetter wrote:
: So now I wonder why BinaryRequestWriter (and BinaryUpdateRequestHandler)
: aren't turned on by default. (eps considering some threads on the dev-list
: So now I wonder why BinaryRequestWriter (and BinaryUpdateRequestHandler)
: aren't turned on by default. (eps considering some threads on the dev-list
I don't really understand this question -- the BinaryUpdateRequestHandler
is registered with the path /update/javabin in the example solrconfig.
Hi all,
I did some further investigation and (after turning of some filters in
yourkit) found that is was actually the machine sending the files to
solr that was slowing things down.
At first I couldn't find this as it turned out that yourkit hides
org.apache.* classes. When I removed this f
: StreamingUpdateSolrServer already has multiple threads and uses multiple
: connections under the covers. At least the api says ' Uses an internal
Hmmm... i think one of us missunderstands the point behind
StreamingUpdateSolrServer and it's internal threads/queues. (it's very
possible that
solr-user@lucene.apache.org
Subject: Re: Machine utilization while indexing
I'm really only guessing here, but based on your description of what you
are doing it sounds like you only have one thread streaming documents to
solr (via a single StreamingUpdateSolrServer instance which creates
I'm really only guessing here, but based on your description of what you
are doing it sounds like you only have one thread streaming documents to
solr (via a single StreamingUpdateSolrServer instance which creates a
single HTTP connection)
Have you at all attempted to have parallel threads in
w.yert.com/film.php
--- On Thu, 5/20/10, Nagelberg, Kallin wrote:
> From: Nagelberg, Kallin
> Subject: RE: Machine utilization while indexing
> To: "'solr-user@lucene.apache.org'"
> Date: Thursday, May 20, 2010, 8:16 AM
> How about throwing a blockingqueue,
&
here.
-Kallin Nagelberg
-Original Message-
From: Thijs [mailto:vonk.th...@gmail.com]
Sent: Thursday, May 20, 2010 11:25 AM
To: solr-user@lucene.apache.org
Subject: Re: Machine utilization while indexing
I already have a blockingqueue in place (that's my custom queue) and
luckily I'm
on [mailto:gear...@sbcglobal.net]
Sent: Thursday, May 20, 2010 11:25 AM
To: solr-user@lucene.apache.org
Subject: RE: Machine utilization while indexing
It takes that long to do indexing? I'm HOPING to have a site that has low 10's
of millions of documents to billions.
Sounds to me like
out impacting your
current queries.
Dennis Gearon
Signature Warning
EARTH has a Right To Life,
otherwise we all die.
Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Thu, 5/20/10, Nagelberg, Kallin wrote:
From: Nagelberg, Kallin
Subject:
I already have a blockingqueue in place (that's my custom queue) and
luckily I'm indexing faster then what your doing.Currently it takes
about 2hour to index the 5m documents I'm talking about. But I still
feel as if my machine is under utilized.
Thijs
On 20-5-2010 17:16, Nagelberg, Kallin w
n
Signature Warning
EARTH has a Right To Life,
otherwise we all die.
Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Thu, 5/20/10, Nagelberg, Kallin wrote:
> From: Nagelberg, Kallin
> Subject: RE: Machine utilization while indexing
> To:
How about throwing a blockingqueue,
http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/BlockingQueue.html,
between your document-creator and solrserver? Give it a size of 10,000 or
something, with one thread trying to feed it, and one thread waiting for it to
get near full then draini
13 matches
Mail list logo
