> I think they get merged in by the merger, ideally in the background. That sounds sensible. (In other words, we wont concern ourselves with roll backs--something possible while a "layer" is still around.)
I've been thinking about this problem also. One approach discussed earlier in these mailing lists has been to somehow maintain a parallel index of the update-able of the fields in such a way that the docIds of the parallel index remain in sync with the "master" index. Mike McCandless and I were discussing some variants of this approach a few months back: http://markmail.org/message/uifz5v37k6qxxhvz?q=%22incremental+document+field+update%22+site:markmail%2Eorg&page=1&refer=ipebtbf24y7rleps That approach involved the concept of mapping (chaining, if you will) internal docIds to view ids. That docid mapping concept sounds analogous to this layer concept we are discussing now. I now think the parallel index approach may not be such a great idea, after all: it simply pushes the problem to the edge--the slave index. If we can solve update problem in the slave index, I reason, then shouldn't we also be able to solve the same update problem in the master index (and thereby remove the necessity of maintaining a (user-level) parallel index in the first place)? Which seems to align with the approach being discussed here.. I imagine the "layers" being discussed here are somehow threaded by docId. That is, given a docId, you can quickly find it's "layers." If so, then the docId mapping idea may be one way to thread these layers. (A logical document would be constructed by a chain of docIds, each overriding the previous for each field it defines (or deletes). Such a construction would have to be "merge-aware" (perhaps using machinery similar to that used in LUCENE-1879) in order that it may maintain the docId chain. What do you think? On Fri, Apr 2, 2010 at 4:56 AM, Grant Ingersoll <gsing...@apache.org> wrote: > > On Apr 2, 2010, at 2:50 AM, Babak Farhang wrote: > >> [Late to this party, but thought I'd chime in] >> >> I think this "layer" concept is right on. But I'm wondering about the >> life cycle of these layers. Do layers live forever? Or do they >> collapse at some point? (Like, as I think was already pointed out, >> deletes are when segments are merged today.) > > I think they get merged in by the merger, ideally in the background. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
