I'm generally OK with merging them in Spark 4.3.0 as long as these commits do not open/extend the half-baked feature. (If you think that's not something to "complete" the feature, please consider more on the justification) - regardless of my comment, I'd leave the final decision to committer+ who signs off those PRs.
On Tue, Aug 11, 2026 at 9:14 AM Yan Yan <[email protected]> wrote: > Hi Jungtaek, > > Apologies for the delay, but I wonder if it's possible to include the > following PRs in the 4.3 release: > - https://github.com/apache/spark/pull/57799/ > - https://github.com/apache/spark/pull/57585 > - https://github.com/apache/spark/pull/57865 > > They are follow up changes to https://github.com/apache/spark/pull/57582 > which got merged around the branch cut time, and they together complete the > support for passing options to the connector during DSv2 read/write > operations while ensuring table version remains consistent throughout a > single query. Some are very close to be merged. > > Thank you! > Yan > > On 2026/08/05 04:52:39 Jungtaek Lim wrote: > > Thanks for the information! I'm OK with including it in Apache Spark > 4.3.0 > > as long as it is self-contained and reviewers also agree with your > > statement. I'll leave the decision to the committer who will perform the > > final sign-off. > > > > On Wed, Aug 5, 2026 at 12:10 PM Anurag Mantripragada < > > [email protected]> wrote: > > > > > Hi Jungtaek, > > > > > > To add some context, this PR is part of the SPIP to introduce new DSv2 > > > capabilities for connectors supporting column-level updates. It is > fully > > > self-contained and enables UPDATE statements, allowing connectors to > opt in > > > and receive only the updated data. This will help connectors like > Iceberg > > > and Delta fully implement column-level UPDATE statements, which are the > > > primary use cases for this feature. I expect just one more JIRA next to > > > support MERGE INTO cases, which can target 4.4. > > > > > > I wanted to bring this to your attention as the release manager since > the > > > PR might be merged soon. I will defer to the decision made by the > reviewer, > > > committer, and you (release manager). > > > > > > Thanks, > > > Anurag > > > > > > > > > On Tue, Aug 4, 2026 at 5:38 PM Jungtaek Lim < > [email protected]> > > > wrote: > > > > > >> Hi Anurag, could you please provide the impact of merging this in > Apache > > >> Spark 4.3.0? > > >> > > >> The SPIP only has one JIRA ticket, which doesn't provide the proper > > >> visibility of where we are. I'd not encourage people to leave the > project > > >> to be released as half-baked and trigger "unknown unknown"; does the > PR > > >> bring up self-contained value? I assume the answer is yes, but let's > punt > > >> to 4.4.0 if the answer is no. > > >> > > >> On Wed, Aug 5, 2026 at 6:47 AM Anurag Mantripragada < > > >> [email protected]> wrote: > > >> > > >>> Hi Jungteak, > > >>> > > >>> Would it be possible to include SPARK-58111 > > >>> <https://issues.apache.org/jira/browse/SPARK-58111> (Which is part > of > > >>> the SPIP SPARK-56599 < > https://issues.apache.org/jira/browse/SPARK-56599>) > > >>> in the 4.3 release? > > >>> > > >>> The PR has gone through several review rounds. Including this > > >>> foundational PR with UPDATE support in 4.3 would allow projects like > Apache > > >>> Iceberg to start integrating sooner. > > >>> > > >>> Please let me know your thoughts. > > >>> > > >>> ~ Anurag > > >>> > > >>> On Mon, Aug 3, 2026 at 9:48 PM Jungtaek Lim < > > >>> [email protected]> wrote: > > >>> > > >>>> Thanks Cheng! I'm unblocked now and have several PRs to proceed with > > >>>> post branch cut tasks. > > >>>> > > >>>> * [SPARK-58534][BUILD][4.x] Change version in branch-4.x to 4.4.0 > (PR > > >>>> <https://github.com/apache/spark/pull/57737>) > > >>>> * [SPARK-58534][BUILD][FOLLOWUP] Add 4.4 MiMa excludes section to > > >>>> master (PR <https://github.com/apache/spark/pull/57738>) > > >>>> * [SPARK-58538][INFRA] Add branch-4.3 CI scheduler and release > > >>>> integration (PR <https://github.com/apache/spark/pull/57741>) > > >>>> * [SPARK-58538][INFRA][4.3] Fix branch-4.3 workflow titles and > default > > >>>> branch (PR <https://github.com/apache/spark/pull/57740>) > > >>>> * [SPARK-58537][INFRA][4.x] Make maven_test.yml default to > branch-4.x > > >>>> on branch-4.x (PR <https://github.com/apache/spark/pull/57739>) > > >>>> > > >>>> Folks, please help reviewing the changes; thanks for your help! > > >>>> > > >>>> > > >>>> On Tue, Aug 4, 2026 at 12:05 PM Cheng Pan <[email protected]> > wrote: > > >>>> > > >>>>> I just merged https://github.com/apache/spark/pull/57718, and I > > >>>>> suppose it fixes the compilation for branch-4.x and branch-4.3 > > >>>>> > > >>>>> Thanks, > > >>>>> Cheng Pan > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Aug 4, 2026, at 10:58, Jungtaek Lim < > [email protected]> > > >>>>> wrote: > > >>>>> > > >>>>> UPDATE: It takes more than I expect to do post-branch cut, because > > >>>>> branch-4.x doesn't run CI and seems broken with maven now (haven't > run sbt > > >>>>> yet). > > >>>>> Maybe I'll need to fix the issue altogether in branch-4.x and > > >>>>> branch-4.3, and then bump branch-4.x to 4.4.0-SNAPSHOT. > > >>>>> > > >>>>> On Tue, Aug 4, 2026 at 10:51 AM Jungtaek Lim < > > >>>>> [email protected]> wrote: > > >>>>> > > >>>>>> UPDATE: the new branch "branch-4.3" is created. Committers/PMC > > >>>>>> members who merge the code change to Apache Spark 4.3.0 would > need to > > >>>>>> include branch-4.3 into the merge branches to account for it. I'm > still > > >>>>>> making the change to update 4.x to refer to 4.4.0-SNAPSHOT; I'll > give an > > >>>>>> update once that is done. > > >>>>>> > > >>>>>> This is a soft cut and we tend to broadly accept code changes to > > >>>>>> branch-4.3 till RC is started, but please refrain from merging > the code > > >>>>>> change which could make a release at risk of releasing in a month > or so. > > >>>>>> Dragging a release process to September 4th week would risk the > > >>>>>> unavailability of PMC members in East Asia for a couple weeks, so > ideally > > >>>>>> it would be nice to finalize the RC vote till Sep 3rd week. > > >>>>>> > > >>>>>> Thanks! > > >>>>>> > > >>>>>> > > >>>>>> On Mon, Aug 3, 2026 at 9:25 AM Jungtaek Lim < > > >>>>>> [email protected]> wrote: > > >>>>>> > > >>>>>>> FYI, this is the last reminder before branch cut on Apache Spark > > >>>>>>> 4.3.0. I'll cut the branch tomorrow (Aug 3 in US tz) if there is > no > > >>>>>>> reasonable objection. I'll give an update tomorrow once I cut > the branch, > > >>>>>>> so that folks have visibility of whether the fix is included or > not. > > >>>>>>> > > >>>>>>> On Tue, Jul 28, 2026 at 7:02 AM Jungtaek Lim < > > >>>>>>> [email protected]> wrote: > > >>>>>>> > > >>>>>>>> UPDATE: The website is updated to reflect the plan. > > >>>>>>>> > > >>>>>>>> https://spark.apache.org/versioning-policy.html > > >>>>>>>> > > >>>>>>>> As a reminder, *Aug 1 is the rough date to cut the branch for > > >>>>>>>> Apache Spark 4.3.0*. Since there is no feedback on the plan so > > >>>>>>>> far, I'd consider this as no issue in proceeding with the plan. > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Fri, Jul 24, 2026 at 5:22 PM Hyukjin Kwon < > [email protected]> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> SGTM > > >>>>>>>>> > > >>>>>>>>> On Fri, 24 Jul 2026 at 12:23, Jungtaek Lim < > > >>>>>>>>> [email protected]> wrote: > > >>>>>>>>> > > >>>>>>>>>> Just a head-up, since there is no major feedback, I'll > proceed to > > >>>>>>>>>> update the release plan as discussed with prior SPIP. > > >>>>>>>>>> > > >>>>>>>>>> Given the practice of release scheduling, the plan for Apache > > >>>>>>>>>> 4.3.0 will be following: > > >>>>>>>>>> > > >>>>>>>>>> * Aug 1st 2026: Code Freeze. Branch Cut > > >>>>>>>>>> * Mid Aug 2026: QA period > > >>>>>>>>>> * Late Aug 2026: RC > > >>>>>>>>>> > > >>>>>>>>>> I'm going to update the versioning policy page with the above. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> On Mon, Jul 20, 2026 at 12:07 PM Jungtaek Lim < > > >>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Hi dev, > > >>>>>>>>>>> > > >>>>>>>>>>> I'm happy to serve the role of the release manager for Apache > > >>>>>>>>>>> Spark 4.3. Since this is the first release we followed the > new policy of 3 > > >>>>>>>>>>> months cadence, there might be some confusion/concern and I > need your help > > >>>>>>>>>>> and understanding. (Thanks in advance for your help and > patience!) > > >>>>>>>>>>> > > >>>>>>>>>>> I'm collecting the projects the workstream has been > happening on > > >>>>>>>>>>> Apache Spark 4.3 (branch-4.x at this moment) - I'll be > focusing on SPIP, > > >>>>>>>>>>> but please give the head-up for your project which has > partially landed to > > >>>>>>>>>>> Apache Spark 4.3 and you are close to the finish line. > > >>>>>>>>>>> > > >>>>>>>>>>> For the projects you are unlikely to complete before branch > cut > > >>>>>>>>>>> (loosely plus 1-2 weeks after branch cut), you are > encouraged to have a > > >>>>>>>>>>> flag to disable the partial change to take effect in Apache > Spark 4.3, and > > >>>>>>>>>>> postpone the finish line to Apache Spark 4.4. The delay of > missing Apache > > >>>>>>>>>>> Spark 4.3 wouldn't be long for you thanks to 3 months > release cadence, and > > >>>>>>>>>>> "be open to postpone the schedule for the project" is a key > success factor > > >>>>>>>>>>> for faster release. Thanks for your understanding! > > >>>>>>>>>>> > > >>>>>>>>>>> I'll update the versioning policy page to reflect the > expected > > >>>>>>>>>>> schedule of Apache Spark 4.3.0 release. Please post here for > any questions > > >>>>>>>>>>> and voices/feedback. > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks! > > >>>>>>>>>>> Jungtaek Lim (HeartSaVioR) > > >>>>>>>>>>> > > >>>>>>>>>>> On Mon, Jul 20, 2026 at 11:38 AM Hyukjin Kwon < > > >>>>>>>>>>> [email protected]> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> Hi all, > > >>>>>>>>>>>> > > >>>>>>>>>>>> I would like to send a gentle reminder as heads up that > > >>>>>>>>>>>> we're going to cut the branch early Aug, 1st to prepare > branch-4.3. In the > > >>>>>>>>>>>> SPIP about releasing, we decided to do the faster release > ones, and this > > >>>>>>>>>>>> would be the very first one. > > >>>>>>>>>>>> > > >>>>>>>>>>>> I know that we just released Spark 4.2. That was > unexpectedly > > >>>>>>>>>>>> delayed some more. I would like to avoid affecting our > first release > > >>>>>>>>>>>> schedule - It has been our chronic issue in history, and I > do think we > > >>>>>>>>>>>> should change this from 4.3 at the first predictable > release, and push it > > >>>>>>>>>>>> forward. Let's get this done and let the releases be > predictable. > > >>>>>>>>>>>> > > >>>>>>>>>>>> PS: would need to update > > >>>>>>>>>>>> https://spark.apache.org/versioning-policy.html for > > >>>>>>>>>>>> details about branchcut, RC and releases. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Cc'ing @Jungtaek Lim <[email protected]> who's going to be > the > > >>>>>>>>>>>> release manager for 4.3. > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>> > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
