Hi Jungtaek, To add some context, this PR is part of the SPIP to introduce new DSv2 capabilities for connectors supporting column-level updates. It is fully self-contained and enables UPDATE statements, allowing connectors to opt in and receive only the updated data. This will help connectors like Iceberg and Delta fully implement column-level UPDATE statements, which are the primary use cases for this feature. I expect just one more JIRA next to support MERGE INTO cases, which can target 4.4.
I wanted to bring this to your attention as the release manager since the PR might be merged soon. I will defer to the decision made by the reviewer, committer, and you (release manager). Thanks, Anurag On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <[email protected]> wrote: > Hi Anurag, could you please provide the impact of merging this in Apache > Spark 4.3.0? > > The SPIP only has one JIRA ticket, which doesn't provide the proper > visibility of where we are. I'd not encourage people to leave the project > to be released as half-baked and trigger "unknown unknown"; does the PR > bring up self-contained value? I assume the answer is yes, but let's punt > to 4.4.0 if the answer is no. > > On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada < > [email protected]> wrote: > >> Hi Jungteak, >> >> Would it be possible to include SPARK-58111 >> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of >> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>) >> in the 4.3 release? >> >> The PR has gone through several review rounds. Including this >> foundational PR with UPDATE support in 4.3 would allow projects like Apache >> Iceberg to start integrating sooner. >> >> Please let me know your thoughts. >> >> ~ Anurag >> >> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim <[email protected]> >> wrote: >> >>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with >>> post branch cut tasks. >>> >>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR >>> <https://github.com/apache/spark/pull/57737>) >>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to master >>> (PR <https://github.com/apache/spark/pull/57738>) >>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release >>> integration (PR <https://github.com/apache/spark/pull/57741>) >>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default >>> branch (PR <https://github.com/apache/spark/pull/57740>) >>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x >>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>) >>> >>> Folks, please help reviewing the changes; thanks for your help! >>> >>> >>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <[email protected]> wrote: >>> >>>> I just merged https://github.com/apache/spark/pull/57718, and I >>>> suppose it fixes the compilation for branch-4.x and branch-4.3 >>>> >>>> Thanks, >>>> Cheng Pan >>>> >>>> >>>> >>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <[email protected]> >>>> wrote: >>>> >>>> UPDATE: It takes more than I expect to do post-branch cut, because >>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt >>>> yet). >>>> Maybe I'll need to fix the issue altogether in branch-4.x and >>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT. >>>> >>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim < >>>> [email protected]> wrote: >>>> >>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC members >>>>> who merge the code change to Apache Spark 4.3.0 would need to include >>>>> branch-4.3 into the merge branches to account for it. I'm still making the >>>>> change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an update once >>>>> that is done. >>>>> >>>>> This is a soft cut and we tend to broadly accept code changes to >>>>> branch-4.3 till RC is started, but please refrain from merging the code >>>>> change which could make a release at risk of releasing in a month or so. >>>>> Dragging a release process to September 4th week would risk the >>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally >>>>> it would be nice to finalize the RC vote till Sep 3rd week. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim < >>>>> [email protected]> wrote: >>>>> >>>>>> FYI, this is the last reminder before branch cut on Apache Spark >>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no >>>>>> reasonable objection. I'll give an update tomorrow once I cut the branch, >>>>>> so that folks have visibility of whether the fix is included or not. >>>>>> >>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> UPDATE: The website is updated to reflect the plan. >>>>>>> >>>>>>> https://spark.apache.org/versioning-policy.html >>>>>>> >>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for >>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so far, >>>>>>> I'd consider this as no issue in proceeding with the plan. >>>>>>> >>>>>>> >>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> SGTM >>>>>>>> >>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to >>>>>>>>> update the release plan as discussed with prior SPIP. >>>>>>>>> >>>>>>>>> Given the practice of release scheduling, the plan for Apache >>>>>>>>> 4.3.0 will be following: >>>>>>>>> >>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut >>>>>>>>> * Mid Aug 2026: QA period >>>>>>>>> * Late Aug 2026: RC >>>>>>>>> >>>>>>>>> I'm going to update the versioning policy page with the above. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi dev, >>>>>>>>>> >>>>>>>>>> I'm happy to serve the role of the release manager for Apache >>>>>>>>>> Spark 4.3. Since this is the first release we followed the new >>>>>>>>>> policy of 3 >>>>>>>>>> months cadence, there might be some confusion/concern and I need >>>>>>>>>> your help >>>>>>>>>> and understanding. (Thanks in advance for your help and patience!) >>>>>>>>>> >>>>>>>>>> I'm collecting the projects the workstream has been happening on >>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on >>>>>>>>>> SPIP, >>>>>>>>>> but please give the head-up for your project which has partially >>>>>>>>>> landed to >>>>>>>>>> Apache Spark 4.3 and you are close to the finish line. >>>>>>>>>> >>>>>>>>>> For the projects you are unlikely to complete before branch cut >>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to >>>>>>>>>> have a >>>>>>>>>> flag to disable the partial change to take effect in Apache Spark >>>>>>>>>> 4.3, and >>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing >>>>>>>>>> Apache >>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release >>>>>>>>>> cadence, and >>>>>>>>>> "be open to postpone the schedule for the project" is a key success >>>>>>>>>> factor >>>>>>>>>> for faster release. Thanks for your understanding! >>>>>>>>>> >>>>>>>>>> I'll update the versioning policy page to reflect the expected >>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any >>>>>>>>>> questions >>>>>>>>>> and voices/feedback. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> Jungtaek Lim (HeartSaVioR) >>>>>>>>>> >>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I would like to send a gentle reminder as heads up that >>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare branch-4.3. >>>>>>>>>>> In the >>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, and >>>>>>>>>>> this >>>>>>>>>>> would be the very first one. >>>>>>>>>>> >>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly >>>>>>>>>>> delayed some more. I would like to avoid affecting our first release >>>>>>>>>>> schedule - It has been our chronic issue in history, and I do think >>>>>>>>>>> we >>>>>>>>>>> should change this from 4.3 at the first predictable release, and >>>>>>>>>>> push it >>>>>>>>>>> forward. Let's get this done and let the releases be predictable. >>>>>>>>>>> >>>>>>>>>>> PS: would need to update >>>>>>>>>>> https://spark.apache.org/versioning-policy.html for >>>>>>>>>>> details about branchcut, RC and releases. >>>>>>>>>>> >>>>>>>>>>> Cc'ing @Jungtaek Lim <[email protected]> who's going to be the >>>>>>>>>>> release manager for 4.3. >>>>>>>>>>> >>>>>>>>>>> >>>>
