On Thu, Apr 8, 2010 at 6:21 PM, Earwin Burrfoot <ear...@gmail.com> wrote: >> But, IW doesn't let you "hold on to" checkpoints... only to commits. >> >> Ie SnapshotDP will only "see" actual commit/close calls, not >> intermediate checkpoints like a random segment merge completing, a >> flush happening, etc. >> >> Or... maybe you would in fact call commit frequently from the main >> threads (but with fsync disabled), and then your DP holds onto these >> "fake commits", periodically picking one of them to do the "real" >> fsync ing? > Yeah, that's exactly what I tried to describe in my initial post :)
Ahh ok then it makes more sense. But still you shouldn't commit that often (even with fake fsync) since it must flush the segment. >>>>> I'm just playing around with stupid idea. I'd like to have NRT >>>>> look-alike without binding readers and writers. :) >>>> I see... well binding durability & visibility will always be costly. >>>> This is why Lucene decouples them (by making NRT readers available). >>> My experiments do the same, essentially. >>> But after I understood that to perform deletions IW has to load term indexes >>> anyway, I'm almost ready to give up and go for intertwined IW/IR mess :) >> Hey if you really think it's a mess, post a patch that cleans it up :) > Uh oh. Let me finish current one, first. Heh, yes :) > Second - I don't know yet how > this should look like. > Something along the lines of deletions/norms writers being extracted > from segment reader > and reader pool being made external to IW?? Yeah, reader pool should be pulled out of IW, and I think IW should be split into "that which manages the segment infos", "that which adds/deletes docs", and "the rest" (merging, addIndexes*)? (There's an issue open for this refactoring...). I'm not sure about deletions/norms writers being extracted from SR.... I think delete ops would still go through IW? Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
