Re: Fastest batch indexing with 1.3-rc1

[Doug Cutting]

As the index grows, disk i/o becomes the bottleneck.  The default 
indexing parameters do a pretty good job of optimizing this.  But if you 
have lots of CPUs and lots of disks, you might try building several 
indexes in parallel, each containing a subset of the documents, optimize 
each index and finally merge them all into a single index at the end. 
But you need lots of i/o capacity for this to pay off.

Doug

Dan Quaroni wrote:
Looks like I spoke too soon... As the index gets larger, time to merge
becomes prohibitably high.  It appears to increase linearly.
Oh well.  I guess I'll just have to go with about 3ms/doc.

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]