+1, I like the idea in general. We will have to work out the details in practice as we come across "index breaking" changes, and where/how to draw the line of "best effort". But I think this is an improvement for our users over the hard check we now have for "only N-1", and likely not so much development effort?
I think where it might get interesting is if we want to make a Codec API change, maybe to optimize a interesting use-cases, and then we must do some development to fix N-2 BWC codec (as well as N-1 BWC codec that we already must fix for such an example, today). Some users seem to keep their indices alive for a very long time! Mike McCandless http://blog.mikemccandless.com On Sat, Jan 9, 2021 at 6:13 AM Simon Willnauer <simon.willna...@gmail.com> wrote: > I can provide some examples of BWC issues and what we would do if it > happened in the future: > > - negative offsets: in this case it would be best effort to add a > wrapper around the older formats to check if the offsets go backwards > on the read side and throw an exception to prevent consumers making > the assumption that offsets go forward only from failing or going OOM > etc. > - norms encoding: in this case it would be best effort in the older > norms formats to convert to the newer encodings. > - the removal of numeric fields queries would not fall under the > promises we make with compatibility of N-2 and it would be the > responsibility of the user to keep the code around that understands > the value of a field. > > I hope this clarifies some of the aspects? > > we would only do all this for the reading end, for writing we would > reject indices that are older than N-1 > > simon > > > On Thu, Jan 7, 2021 at 8:04 PM jim ferenczi <jim.feren...@gmail.com> > wrote: > > > > The proposal is only about keeping the ability to read file-format up to > N-2. Everything that is done on top of the file format is not guaranteed > and should be supported on a best-effort basis. > > That's an important aspect if we don't want to block innovation. So in > practice that means that queries that require some specific file format or > analyzers that change behaviors in major versions would not be part of the > extended guarantee. > > > > > > Le mer. 6 janv. 2021 à 21:53, Yonik Seeley <ysee...@gmail.com> a écrit : > >> > >> On Wed, Jan 6, 2021 at 4:40 AM Simon Willnauer < > simon.willna...@gmail.com> wrote: > >>> > >>> You can open a reader on an index created by > >>> version N-2, but you cannot open an IndexWriter on it > >> > >> > >> +1 > >> There should definitely be more consideration given to back compat in > general... it's caused a ton of pain to users over time. > >> > >> -Yonik > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
