Grant, Reading your post a 3rd time, I see my "suggestion" is in fact the approach you describe. Sorry for being redundant.
-Babak On Fri, Apr 2, 2010 at 11:25 PM, Babak Farhang <farh...@gmail.com> wrote: >> I think they get merged in by the merger, ideally in the background. > > That sounds sensible. (In other words, we wont concern ourselves with > roll backs--something possible while a "layer" is still around.) > > I've been thinking about this problem also. One approach discussed > earlier in these mailing lists has been to somehow maintain a parallel > index of the update-able of the fields in such a way that the docIds > of the parallel index remain in sync with the "master" index. Mike > McCandless and I were discussing some variants of this approach a few > months back: > http://markmail.org/message/uifz5v37k6qxxhvz?q=%22incremental+document+field+update%22+site:markmail%2Eorg&page=1&refer=ipebtbf24y7rleps > That approach involved the concept of mapping (chaining, if you will) > internal docIds to view ids. That docid mapping concept sounds > analogous to this layer concept we are discussing now. > > I now think the parallel index approach may not be such a great idea, > after all: it simply pushes the problem to the edge--the slave index. > If we can solve update problem in the slave index, I reason, then > shouldn't we also be able to solve the same update problem in the > master index (and thereby remove the necessity of maintaining a > (user-level) parallel index in the first place)? > > Which seems to align with the approach being discussed here.. > > I imagine the "layers" being discussed here are somehow threaded by > docId. That is, given a docId, you can quickly find it's "layers." If > so, then the docId mapping idea may be one way to thread these layers. > (A logical document would be constructed by a chain of docIds, each > overriding the previous for each field it defines (or deletes). Such > a construction would have to be "merge-aware" (perhaps using machinery > similar to that used in LUCENE-1879) in order that it may maintain the > docId chain. > > What do you think? > > > On Fri, Apr 2, 2010 at 4:56 AM, Grant Ingersoll <gsing...@apache.org> wrote: >> >> On Apr 2, 2010, at 2:50 AM, Babak Farhang wrote: >> >>> [Late to this party, but thought I'd chime in] >>> >>> I think this "layer" concept is right on. But I'm wondering about the >>> life cycle of these layers. Do layers live forever? Or do they >>> collapse at some point? (Like, as I think was already pointed out, >>> deletes are when segments are merged today.) >> >> I think they get merged in by the merger, ideally in the background. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
