On Thursday 23 December 2004 00:45, Ryan Aslett wrote:
> When all machines and all threads are finished, I should have a slew of
> index slices that I want to combine together to create one index.
You should simply skip this step and instead search the small indices with
a ParallelMultiSearcher.
d only open the
connection to the live index machine when necessary
Hope this helps
Garrett
-Original Message-
From: Ryan Aslett [mailto:[EMAIL PROTECTED]
Sent: 22 December 2004 23:45
To: Lucene Users List
Subject: addIndexes() Question
Hi there, Im about to embark on a Lucene proj
I think you should change a little bit your plans, and to think that
your goal is to
create a fast search engine not a fast indexing engine.
When you plan to index a lot of documents then it is possible to creata
a lot of segments (if you don't optimize the index)
and the serch will be very slow
I _think_ you'd be better off doing it all at once, but I wouldn't
trust myself on this and would instead construct a small 3-index set
and test, looking at a) maximal disk usage, b) time, and c) RAM usage.
:)
Otis
--- Ryan Aslett <[EMAIL PROTECTED]> wrote:
>
> Hi there, Im about to embark on
Hi there, Im about to embark on a Lucene project of massive scale
(between 500 million and 2 billion documents). I am currently working
on parallellizing the construction of the Index(es).
Rough summary of my plan:
I have many, many physical machines, each with multiple processors that
I wish
