Late joining... could we agree on an "intention" to provide an index migration tool when/if format back comp. has to be broken? It is not clear to me that this was agreed... So here is a suggestion for a revised index format backwards compatibility policy:
Starting release 4.0, Lucene has a limited file formats back-compatibility between major versions, falling into one of the three possible levels: (Level 1) When possible, Version X is would be able to read indexes generated by any X-1 version after and including version X-1.0. (Level 2) If version X cannot read indexes of version X-1, the release of version X would be accompanied by a tool for migrating indexes from X-1 to X, unless (Level 3) the nature of the specific change does not allow for the development of such a migration tool. For the exact level of file back compatibility of a release see the specific release notes. Not sure if the "unless" part (no. 3) would ever materialize, but I think it provides a required freedom. Doron