Hi Yixun, Given that Lucene supports file format backwards compatibility for X-1 major version release with version X application code, https://cwiki.apache.org/confluence/display/lucene/BackwardsCompatibility
Here is something you can try: Keep using Lucene 8 indexes and Lucene 8 application code as your baseline. Then switch just the application code to Lucene 9 (this will presumably be slightly different as some APIs must've changed) If something goes wrong, just switch back to Lucene 8 application code, else, just keep using Lucene 9 code. Then, switch to Lucene 9 indexes, and they should just work with Lucene 9 application code (I have some small concerns here around whether or not the exact same Lucene APIs used to read Lucene 8 indexes would work with Lucene 9 indexes, if not the application code would change here slightly). I would first perform these steps in just a single dev box with a static index and check the flow. Hope this helps. On Sat, Jun 24, 2023 at 8:28 AM Yixun Xu <yix...@gmail.com> wrote: > Thank you for the suggestion. We do keep the original data around in an > upstream database, so we are also able to reindex the data after the > rollback, but I'm curious if there are ways to avoid the need to reindex > for some time. We'll eventually start using the new codecs for sure, at > which time we'll lose the ability to rollback without reindex anyways, but > ideally I'd like to make that a separate step from the Lucene upgrade, and > possibly apply the codec upgrade gradually per index. > > Best regards, > Yixun Xu > > > On Fri, Jun 23, 2023 at 8:19 PM Steven Schlansker < > stevenschlans...@gmail.com> wrote: > > > Hi, > > > > > On Jun 23, 2023, at 2:34 PM, Yixun Xu <yix...@gmail.com> wrote: > > > > > > Hello, > > > > > > I have a service that creates and manages Lucene indices. The service > is > > > using Lucene 8 and I want to upgrade to Lucene 9, and I would like to > be > > > able to rollback the upgrade in case I encounter any issues (*). The > > > problem is that the older version of the service won't be able to read > > the > > > segments created after the upgrade because they would be created using > > > Lucene 9's codec. > > > > > > One potential solution is to keep writing new indices using Lucene 8's > > > codecs after upgrading to Lucene 9, and only start using Lucene 9's > codec > > > after some time. However this doesn't seem to be officially supported: > > the > > > lucene-backward-codecs package provides older codecs, but they are > > > read-only. > > > > > > My questions are: > > > 1. I can try copying over the actual codecs from Lucene 8 instead of > > using > > > lucene-backward-codecs, but is this actually supposed to work, or will > > > there be undefined behaviors when the codec is used with Lucene 9? > > > 2. Is there an alternative solution to support rollbacks? > > > > This is probably not the solution you are looking for, but the way we > > manage this is to > > have a higher level logical version (including Lucene codes, > > backwards-incompatible document format changes, etc). > > We read all indexing operations off of a Kafka topic, and each index > > format version independently keeps Kafka offsets. > > So, after upgrading to index format X+1, rolling back to index format X > > will also rewind the Kafka consumers and they > > will replay any missed indexing operations immediately. > > > > > > > > Any help would be greatly appreciated! > > > > > > Best regards, > > > Yixun Xu > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > >
