I agree: we should figure out just how an app would effectively make use of this seq ID, in order to understand if this really is gonna "work" end to end. Else we shouldn't change Lucene's core APIs.
EG: could ES remove its lock array if Lucene returned a seq ID? How "bad" is it that ES/Solr/this-new-module would have to order their transaction log according to Lucene's seq ID? Or maybe it would not re-order, but rather write the seqID+document in each entry; then on playback (but also on RT get) it'd have to re-order? Mike McCandless http://blog.mikemccandless.com On Sat, Sep 10, 2011 at 1:45 PM, Simon Willnauer <[email protected]> wrote: > On Thu, Sep 8, 2011 at 5:35 PM, Yonik Seeley <[email protected]> > wrote: >> On Thu, Sep 8, 2011 at 11:26 AM, Michael McCandless >> <[email protected]> wrote: >>> Returning a long seqID seems the least invasive change to make this >>> total ordering possible? Especially since the DWDQ already computes >>> this order... >> >> +1 >> This seems like the most powerful option. > > I still wonder how we make efficient use of this. If you are ordering > the logs based on the returned sequence Ids you have to effectively > delay writing to the log since documents ie. their threads come back > async and out of order. Even worse if some thread picks up a flush it > might block for a reasonable amount of time. I am not saying its > impossible but before we jump on it and get into the DWPT hassle we > should at least sketch out how to make use of this feature (lemme tell > you this is not trivial to implement and requires a fair bit of > refactoring). If somebody has thought about this I'd be happy if you > could share you ideas here! > > simon >> >> -Yonik >> http://www.lucene-eurocon.com - The Lucene/Solr User Conference >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
