I was kind of wondering what magic had been done to achieve multiple writing to
the index file :-)
BTW, wouldn't it be possible to have seperate segments per thread? Set up the
index with a minimum (desired?) segment count, and write each individually?
Is there any organization in the segments? Or can adjacent data be found in
different segments?
I seem to remember that the new stuff gets committed to its own segment until
some sort of 'consolidate' command takes place.
Dennis Gearon
Signature Warning
EARTH has a Right To Life,
otherwise we all die.
Read 'Hot, Flat, and Crowded'
Laugh at http://www.yert.com/film.php
--- On Thu, 9/23/10, Jan Høydahl / Cominvent wrote:
> From: Jan Høydahl / Cominvent
> Subject: Re: is indexing single-threaded?
> To: solr-user@lucene.apache.org
> Date: Thursday, September 23, 2010, 1:42 AM
> SolrJ threads speeds up feeding
> throughput. The building the index is still single threaded
> (per core), isn't it? Don't know about analysis. But you
> cannot have two threads write to the same file...
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> On 23. sep. 2010, at 08.01, Ryan McKinley wrote:
>
> > Multiple threads work well.
> >
> > If you are using solrj, check the StreamingSolrServer
> for an
> > implementation that will keep X number of threads
> busy.
> >
> > Your mileage will very, but in general I find a
> reasonable thread
> > count is ~ (number of cores)+1
> >
> >
> > On Wed, Sep 22, 2010 at 5:52 AM, Andy
> wrote:
> >> Does Solr index data in a single thread or can
> data be indexed concurrently in multiple threads?
> >>
> >> Thanks
> >> Andy
> >>
> >>
> >>
> >>
>
>
