Hi Jungtaek, Apologies for the delay, but I wonder if it's possible to include the following PRs in the 4.3 release: - https://github.com/apache/spark/pull/57799/ - https://github.com/apache/spark/pull/57585 - https://github.com/apache/spark/pull/57865
They are follow up changes to https://github.com/apache/spark/pull/57582 which got merged around the branch cut time, and they together complete the support for passing options to the connector during DSv2 read/write operations while ensuring table version remains consistent throughout a single query. Some are very close to be merged. Thank you! Yan On 2026/08/05 04:52:39 Jungtaek Lim wrote: > Thanks for the information! I'm OK with including it in Apache Spark 4.3.0 > as long as it is self-contained and reviewers also agree with your > statement. I'll leave the decision to the committer who will perform the > final sign-off. > > On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada < > [email protected]> wrote: > > > Hi Jungtaek, > > > > To add some context, this PR is part of the SPIP to introduce new DSv2 > > capabilities for connectors supporting column-level updates. It is fully > > self-contained and enables UPDATE statements, allowing connectors to opt in > > and receive only the updated data. This will help connectors like Iceberg > > and Delta fully implement column-level UPDATE statements, which are the > > primary use cases for this feature. I expect just one more JIRA next to > > support MERGE INTO cases, which can target 4.4. > > > > I wanted to bring this to your attention as the release manager since the > > PR might be merged soon. I will defer to the decision made by the reviewer, > > committer, and you (release manager). > > > > Thanks, > > Anurag > > > > > > On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <[email protected]> > > wrote: > > > >> Hi Anurag, could you please provide the impact of merging this in Apache > >> Spark 4.3.0? > >> > >> The SPIP only has one JIRA ticket, which doesn't provide the proper > >> visibility of where we are. I'd not encourage people to leave the project > >> to be released as half-baked and trigger "unknown unknown"; does the PR > >> bring up self-contained value? I assume the answer is yes, but let's punt > >> to 4.4.0 if the answer is no. > >> > >> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada < > >> [email protected]> wrote: > >> > >>> Hi Jungteak, > >>> > >>> Would it be possible to include SPARK-58111 > >>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of > >>> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>) > >>> in the 4.3 release? > >>> > >>> The PR has gone through several review rounds. Including this > >>> foundational PR with UPDATE support in 4.3 would allow projects like > >>> Apache > >>> Iceberg to start integrating sooner. > >>> > >>> Please let me know your thoughts. > >>> > >>> ~ Anurag > >>> > >>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim < > >>> [email protected]> wrote: > >>> > >>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with > >>>> post branch cut tasks. > >>>> > >>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR > >>>> <https://github.com/apache/spark/pull/57737>) > >>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to > >>>> master (PR <https://github.com/apache/spark/pull/57738>) > >>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release > >>>> integration (PR <https://github.com/apache/spark/pull/57741>) > >>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default > >>>> branch (PR <https://github.com/apache/spark/pull/57740>) > >>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x > >>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>) > >>>> > >>>> Folks, please help reviewing the changes; thanks for your help! > >>>> > >>>> > >>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <[email protected]> wrote: > >>>> > >>>>> I just merged https://github.com/apache/spark/pull/57718, and I > >>>>> suppose it fixes the compilation for branch-4.x and branch-4.3 > >>>>> > >>>>> Thanks, > >>>>> Cheng Pan > >>>>> > >>>>> > >>>>> > >>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <[email protected]> > >>>>> wrote: > >>>>> > >>>>> UPDATE: It takes more than I expect to do post-branch cut, because > >>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run > >>>>> sbt > >>>>> yet). > >>>>> Maybe I'll need to fix the issue altogether in branch-4.x and > >>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT. > >>>>> > >>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim < > >>>>> [email protected]> wrote: > >>>>> > >>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC > >>>>>> members who merge the code change to Apache Spark 4.3.0 would need to > >>>>>> include branch-4.3 into the merge branches to account for it. I'm still > >>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give > >>>>>> an > >>>>>> update once that is done. > >>>>>> > >>>>>> This is a soft cut and we tend to broadly accept code changes to > >>>>>> branch-4.3 till RC is started, but please refrain from merging the code > >>>>>> change which could make a release at risk of releasing in a month or > >>>>>> so. > >>>>>> Dragging a release process to September 4th week would risk the > >>>>>> unavailability of PMC members in East Asia for a couple weeks, so > >>>>>> ideally > >>>>>> it would be nice to finalize the RC vote till Sep 3rd week. > >>>>>> > >>>>>> Thanks! > >>>>>> > >>>>>> > >>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> FYI, this is the last reminder before branch cut on Apache Spark > >>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no > >>>>>>> reasonable objection. I'll give an update tomorrow once I cut the > >>>>>>> branch, > >>>>>>> so that folks have visibility of whether the fix is included or not. > >>>>>>> > >>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim < > >>>>>>> [email protected]> wrote: > >>>>>>> > >>>>>>>> UPDATE: The website is updated to reflect the plan. > >>>>>>>> > >>>>>>>> https://spark.apache.org/versioning-policy.html > >>>>>>>> > >>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for > >>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so > >>>>>>>> far, I'd consider this as no issue in proceeding with the plan. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <[email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> SGTM > >>>>>>>>> > >>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim < > >>>>>>>>> [email protected]> wrote: > >>>>>>>>> > >>>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to > >>>>>>>>>> update the release plan as discussed with prior SPIP. > >>>>>>>>>> > >>>>>>>>>> Given the practice of release scheduling, the plan for Apache > >>>>>>>>>> 4.3.0 will be following: > >>>>>>>>>> > >>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut > >>>>>>>>>> * Mid Aug 2026: QA period > >>>>>>>>>> * Late Aug 2026: RC > >>>>>>>>>> > >>>>>>>>>> I'm going to update the versioning policy page with the above. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim < > >>>>>>>>>> [email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi dev, > >>>>>>>>>>> > >>>>>>>>>>> I'm happy to serve the role of the release manager for Apache > >>>>>>>>>>> Spark 4.3. Since this is the first release we followed the new > >>>>>>>>>>> policy of 3 > >>>>>>>>>>> months cadence, there might be some confusion/concern and I need > >>>>>>>>>>> your help > >>>>>>>>>>> and understanding. (Thanks in advance for your help and patience!) > >>>>>>>>>>> > >>>>>>>>>>> I'm collecting the projects the workstream has been happening on > >>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing > >>>>>>>>>>> on SPIP, > >>>>>>>>>>> but please give the head-up for your project which has partially > >>>>>>>>>>> landed to > >>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line. > >>>>>>>>>>> > >>>>>>>>>>> For the projects you are unlikely to complete before branch cut > >>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to > >>>>>>>>>>> have a > >>>>>>>>>>> flag to disable the partial change to take effect in Apache Spark > >>>>>>>>>>> 4.3, and > >>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of > >>>>>>>>>>> missing Apache > >>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release > >>>>>>>>>>> cadence, and > >>>>>>>>>>> "be open to postpone the schedule for the project" is a key > >>>>>>>>>>> success factor > >>>>>>>>>>> for faster release. Thanks for your understanding! > >>>>>>>>>>> > >>>>>>>>>>> I'll update the versioning policy page to reflect the expected > >>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any > >>>>>>>>>>> questions > >>>>>>>>>>> and voices/feedback. > >>>>>>>>>>> > >>>>>>>>>>> Thanks! > >>>>>>>>>>> Jungtaek Lim (HeartSaVioR) > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon < > >>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Hi all, > >>>>>>>>>>>> > >>>>>>>>>>>> I would like to send a gentle reminder as heads up that > >>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare > >>>>>>>>>>>> branch-4.3. In the > >>>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, > >>>>>>>>>>>> and this > >>>>>>>>>>>> would be the very first one. > >>>>>>>>>>>> > >>>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly > >>>>>>>>>>>> delayed some more. I would like to avoid affecting our first > >>>>>>>>>>>> release > >>>>>>>>>>>> schedule - It has been our chronic issue in history, and I do > >>>>>>>>>>>> think we > >>>>>>>>>>>> should change this from 4.3 at the first predictable release, > >>>>>>>>>>>> and push it > >>>>>>>>>>>> forward. Let's get this done and let the releases be predictable. > >>>>>>>>>>>> > >>>>>>>>>>>> PS: would need to update > >>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for > >>>>>>>>>>>> details about branchcut, RC and releases. > >>>>>>>>>>>> > >>>>>>>>>>>> Cc'ing @Jungtaek Lim <[email protected]> who's going to be the > >>>>>>>>>>>> release manager for 4.3. > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>> > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
