Thanks for the information! I'm OK with including it in Apache Spark 4.3.0 as long as it is self-contained and reviewers also agree with your statement. I'll leave the decision to the committer who will perform the final sign-off.
On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada < [email protected]> wrote: > Hi Jungtaek, > > To add some context, this PR is part of the SPIP to introduce new DSv2 > capabilities for connectors supporting column-level updates. It is fully > self-contained and enables UPDATE statements, allowing connectors to opt in > and receive only the updated data. This will help connectors like Iceberg > and Delta fully implement column-level UPDATE statements, which are the > primary use cases for this feature. I expect just one more JIRA next to > support MERGE INTO cases, which can target 4.4. > > I wanted to bring this to your attention as the release manager since the > PR might be merged soon. I will defer to the decision made by the reviewer, > committer, and you (release manager). > > Thanks, > Anurag > > > On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim <[email protected]> > wrote: > >> Hi Anurag, could you please provide the impact of merging this in Apache >> Spark 4.3.0? >> >> The SPIP only has one JIRA ticket, which doesn't provide the proper >> visibility of where we are. I'd not encourage people to leave the project >> to be released as half-baked and trigger "unknown unknown"; does the PR >> bring up self-contained value? I assume the answer is yes, but let's punt >> to 4.4.0 if the answer is no. >> >> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada < >> [email protected]> wrote: >> >>> Hi Jungteak, >>> >>> Would it be possible to include SPARK-58111 >>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part of >>> the SPIP SPARK-56599 <https://issues.apache.org/jira/browse/SPARK-56599>) >>> in the 4.3 release? >>> >>> The PR has gone through several review rounds. Including this >>> foundational PR with UPDATE support in 4.3 would allow projects like Apache >>> Iceberg to start integrating sooner. >>> >>> Please let me know your thoughts. >>> >>> ~ Anurag >>> >>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim < >>> [email protected]> wrote: >>> >>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with >>>> post branch cut tasks. >>>> >>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 (PR >>>> <https://github.com/apache/spark/pull/57737>) >>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to >>>> master (PR <https://github.com/apache/spark/pull/57738>) >>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release >>>> integration (PR <https://github.com/apache/spark/pull/57741>) >>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and default >>>> branch (PR <https://github.com/apache/spark/pull/57740>) >>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to branch-4.x >>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>) >>>> >>>> Folks, please help reviewing the changes; thanks for your help! >>>> >>>> >>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <[email protected]> wrote: >>>> >>>>> I just merged https://github.com/apache/spark/pull/57718, and I >>>>> suppose it fixes the compilation for branch-4.x and branch-4.3 >>>>> >>>>> Thanks, >>>>> Cheng Pan >>>>> >>>>> >>>>> >>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim <[email protected]> >>>>> wrote: >>>>> >>>>> UPDATE: It takes more than I expect to do post-branch cut, because >>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't run sbt >>>>> yet). >>>>> Maybe I'll need to fix the issue altogether in branch-4.x and >>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT. >>>>> >>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim < >>>>> [email protected]> wrote: >>>>> >>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC >>>>>> members who merge the code change to Apache Spark 4.3.0 would need to >>>>>> include branch-4.3 into the merge branches to account for it. I'm still >>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll give an >>>>>> update once that is done. >>>>>> >>>>>> This is a soft cut and we tend to broadly accept code changes to >>>>>> branch-4.3 till RC is started, but please refrain from merging the code >>>>>> change which could make a release at risk of releasing in a month or so. >>>>>> Dragging a release process to September 4th week would risk the >>>>>> unavailability of PMC members in East Asia for a couple weeks, so ideally >>>>>> it would be nice to finalize the RC vote till Sep 3rd week. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> FYI, this is the last reminder before branch cut on Apache Spark >>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is no >>>>>>> reasonable objection. I'll give an update tomorrow once I cut the >>>>>>> branch, >>>>>>> so that folks have visibility of whether the fix is included or not. >>>>>>> >>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> UPDATE: The website is updated to reflect the plan. >>>>>>>> >>>>>>>> https://spark.apache.org/versioning-policy.html >>>>>>>> >>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for >>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so >>>>>>>> far, I'd consider this as no issue in proceeding with the plan. >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> SGTM >>>>>>>>> >>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Just a head-up, since there is no major feedback, I'll proceed to >>>>>>>>>> update the release plan as discussed with prior SPIP. >>>>>>>>>> >>>>>>>>>> Given the practice of release scheduling, the plan for Apache >>>>>>>>>> 4.3.0 will be following: >>>>>>>>>> >>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut >>>>>>>>>> * Mid Aug 2026: QA period >>>>>>>>>> * Late Aug 2026: RC >>>>>>>>>> >>>>>>>>>> I'm going to update the versioning policy page with the above. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi dev, >>>>>>>>>>> >>>>>>>>>>> I'm happy to serve the role of the release manager for Apache >>>>>>>>>>> Spark 4.3. Since this is the first release we followed the new >>>>>>>>>>> policy of 3 >>>>>>>>>>> months cadence, there might be some confusion/concern and I need >>>>>>>>>>> your help >>>>>>>>>>> and understanding. (Thanks in advance for your help and patience!) >>>>>>>>>>> >>>>>>>>>>> I'm collecting the projects the workstream has been happening on >>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be focusing on >>>>>>>>>>> SPIP, >>>>>>>>>>> but please give the head-up for your project which has partially >>>>>>>>>>> landed to >>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line. >>>>>>>>>>> >>>>>>>>>>> For the projects you are unlikely to complete before branch cut >>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are encouraged to >>>>>>>>>>> have a >>>>>>>>>>> flag to disable the partial change to take effect in Apache Spark >>>>>>>>>>> 4.3, and >>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of missing >>>>>>>>>>> Apache >>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months release >>>>>>>>>>> cadence, and >>>>>>>>>>> "be open to postpone the schedule for the project" is a key success >>>>>>>>>>> factor >>>>>>>>>>> for faster release. Thanks for your understanding! >>>>>>>>>>> >>>>>>>>>>> I'll update the versioning policy page to reflect the expected >>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for any >>>>>>>>>>> questions >>>>>>>>>>> and voices/feedback. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> Jungtaek Lim (HeartSaVioR) >>>>>>>>>>> >>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> I would like to send a gentle reminder as heads up that >>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare >>>>>>>>>>>> branch-4.3. In the >>>>>>>>>>>> SPIP about releasing, we decided to do the faster release ones, >>>>>>>>>>>> and this >>>>>>>>>>>> would be the very first one. >>>>>>>>>>>> >>>>>>>>>>>> I know that we just released Spark 4.2. That was unexpectedly >>>>>>>>>>>> delayed some more. I would like to avoid affecting our first >>>>>>>>>>>> release >>>>>>>>>>>> schedule - It has been our chronic issue in history, and I do >>>>>>>>>>>> think we >>>>>>>>>>>> should change this from 4.3 at the first predictable release, and >>>>>>>>>>>> push it >>>>>>>>>>>> forward. Let's get this done and let the releases be predictable. >>>>>>>>>>>> >>>>>>>>>>>> PS: would need to update >>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for >>>>>>>>>>>> details about branchcut, RC and releases. >>>>>>>>>>>> >>>>>>>>>>>> Cc'ing @Jungtaek Lim <[email protected]> who's going to be the >>>>>>>>>>>> release manager for 4.3. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>
