Hi Kevin, I believe the general practice is to implement a feature in the latest supported version (currently Spark 4.0). Once the PR is merged, the author may choose to backport the feature to older supported versions, but is not obligated to. If the author does not backport it, others who want the feature in an older version can choose to backport it.
Sometimes a change is simple enough that it makes sense to implement it for all supported versions at once (in one PR). In addition, if a change requires changes in core Iceberg that then requires the same change in other Spark versions, the change is implemented for all Spark versions in one PR. Sometimes a feature depends on changes in the latest supported Spark version and so cannot be backported. Finally, sometimes a PR has already been in progress for a long time and the latest supported Spark version changes in the meantime. It may still get merged and then be forward ported. I understand that your intent is to ensure that features/fixes that *can* be backported be backported. A diff of git logs by itself cannot tell you if a missing change is portable or not. How and when do you propose to do this diff, and does the result of the diff cause anything to be blocked or any action to be taken? Do you perhaps envision this to be done as a kind of pre-release audit (with enough time to address missing backports)? - Wing Yew On Thu, Jul 10, 2025 at 6:43 PM Kevin Liu <kevinjq...@apache.org> wrote: > Hi everyone, > > We currently maintain 3 different versions of spark under > https://github.com/apache/iceberg/tree/main/spark > I've seen this issue a couple of times where a feature would be > implemented for only one of the Spark versions. For example, see > https://github.com/apache/iceberg/pull/13324 and > https://github.com/apache/iceberg/pull/13459. It's hard to remember that > there are 3 different versions of Spark. > > Do we want to verify that features are implemented across all 3 versions > if possible? If so, we can diff the git logs between spark 3.4 > <https://github.com/apache/iceberg/commits/main/spark/v3.4>, 3.5 > <https://github.com/apache/iceberg/commits/main/spark/v3.5>, and 4.0 > <https://github.com/apache/iceberg/commits/main/spark/v4.0>. > > Best, > Kevin Liu >
