On 2/21/07, Doron Cohen <[EMAIL PROTECTED]> wrote:
Ning Li wrote:
> There are three main challenges in enabling concurrent merge:
> 1 a robust merge policy
> 2 detect when merge lags document additions/deletions
> 3 how to slow down document additions/deletions (and amortize
> the cost) when merge falls behind
I wonder what it means for current API semantics -
- An application today can set max-bufferred-docs to N, and after
the Nth (or N+1th?) call to addDoc returns, a newly opened searcher
would see these docs. With merges in a background thread this
might not hold.
- Today, after add(), an application can call flush() or close(),
but with a background merge thread these calls would be blocked.
Mmm... this is probably not a behavior change, because today
these operations can trigger a merge that would take a long(er) time.
We shouldn't advertise or guarantee that behavior. This wasn't even
true before the new merge policy was implemented.
- numRamDocs() and ramSizeInBytes() - not sure what they mean
once a background merge thread had started.
IMO, for the current "batch" of documents being buffered.
The "old" buffered documents should be flushed to disk ASAP.
Still, having non blocking adds is compelling.
Somewhat... It would result in some performance increase...
overlapping analysis of new documents with merging of other segments,
resulting in a higher CPU utilization (esp on multi-processor
systems). The larger the maxBufferedDocs, the better.
The downside is another complexity increase though.
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
