dreamer-89 opened a new issue, #12391: URL: https://github.com/apache/lucene/issues/12391
### Description ### Requirement Allow write capabilities with previous major bwc lucene versions to suport rolling upgrades[2] on OpenSearch. ### Background Customers using OpenSearch performs upgrades to move to latest version of OpenSearch software. One of the possible upgrades is rolling upgrade[2] where each node is upgraded one at a time. This upgrade process result in intermediate state where few nodes are on latest OpenSearch version while others are still running on older version. With segment replication[1] feature, the primary shard copies over the segment files onto replica shard copies and works fine when all nodes are running on same version. But, during upgrades it is possible that replica shard copy be running on a older version node and thus, does not understand the segment files written with newer codec on primary shard. This results in replica shard failures and hit search availability. ### Solution attempted and issue OpenSearch attempted to solve mixed cluster issue by updating primary shard copies to keep using older codec until all replica copies are on latest software. This resulted in segment files written with older codec which replica shard can read. This works for upgrades where there is minor Lucene version bump but not when there is major Lucene bump. We identified from [4] and [manual test](https://github.com/opensearch-project/OpenSearch/issues/3881#issuecomment-1581689082) that Lucene moves older codecs into bwc-codecs with intentation of providing reads. Thus, this solution will not work for major Lucene version upgrades because older codecs are maintained as bwc-codecs and does not allow write capabilities. ### References [1] [Segment replication in OpenSearch ](https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/segment-replication/index/) [2] [Rolling upgrades in OpenSearch ](https://opensearch.org/docs/latest/install-and-configure/upgrade-opensearch/rolling-upgrade/) [3] [Issue on OpenSearch](https://github.com/opensearch-project/OpenSearch/issues/3881) [4] [Backward codecs in Lucene ](https://github.com/apache/lucene/tree/main/lucene/backward-codecs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
