Adrien Grand created LUCENE-9234:
------------------------------------
Summary: Keep write support for old codecs?
Key: LUCENE-9234
URL: https://issues.apache.org/jira/browse/LUCENE-9234
Project: Lucene - Core
Issue Type: Wish
Reporter: Adrien Grand
Currenty we maintain read/write support for the latest codec in lucene/core,
and read-only support for codecs of previous versions (up to \{N-1\}.0}) in
lucene/backward-codecs. We often keep write support in test-framework for
testing purposes only.
This raises challenges for Elasticsearch with regard to rolling upgrades: we
have some users who index very large amounts of data on clusters that are quite
large, so that rolling upgrades take significant time. Meanwhile, several
indices may be created.
Allocating indices when the cluster has nodes of different versions requires
care as Lucene indices created on nodes with a newer version cannot be read by
the nodes running the older version. It is possible to force primary replicas
to be allocated on the older nodes, but this brings other problems like
availability, uneven disk usage across nodes, or moving a lot of data around.
If Lucene could write data using the minimum version that exists in the
cluster, this would avoid this problem as the written data could be read by any
node of the cluster. I understand this change would not come for free,
especially when it comes to testing as we'd need to make sure that older Lucene
versions can read indices created by this "compatibility mode".
I'd be curious to understand whether this is a problem for Solr too, if not how
this problem is being handled, and maybe whether there are other problems that
you have encountered that would also benefit from the ability to write data
with an older format.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
