A quick update. - Wenchen figured out a better set of interfaces for supporting views and the PR will be merged any time now: https://github.com/apache/spark/pull/56606 - I am still iterating on SPARK-57544 in https://github.com/apache/spark/pull/56619. I will update the change to address feedback tomorrow.
чт, 18 черв. 2026 р. о 15:59 Anton Okolnychyi <[email protected]> пише: > Wenchen and I are chatting about TableViewCatalog offline right now. We > will try one idea but it is not clear if it is going to work yet. > > I also found that the column ID work that we planned to ship in 4.2 has > some issues w.r.t to nested columns. If released, we will have to maintain > this behavior contract / API forever. I submitted SPARK-57544 for this and > will share a WIP PR later today. > > - Anton > > чт, 18 черв. 2026 р. о 15:07 huaxin gao <[email protected]> пише: > >> Thanks Anton. I’ll wait for your update before cutting RC4. Please let me >> know if you find anything that should block the release. >> >> Best, >> >> Huaxin >> >> On Wed, Jun 17, 2026 at 4:24 PM Anton Okolnychyi <[email protected]> >> wrote: >> >>> After reviewing Iceberg's PR for integrating Spark 4.2, I have some >>> questions about the recently added TableViewCatalog. I am trying to >>> understand if it changes the contract / behavior for connectors. Will >>> report back here tomorrow. No new issues detected so far. >>> >>> вт, 16 черв. 2026 р. о 17:08 huaxin gao <[email protected]> пише: >>> >>>> Hi all, >>>> >>>> Thanks Wenchen for reviewing RC3 and raising the API doc issue. >>>> >>>> Given the -1, I will cancel RC3 and prepare RC4 after the API doc fix >>>> is merged and backported to branch-4.2. I plan to cut RC4 on Thursday. >>>> >>>> Thanks, >>>> Huaxin >>>> >>>> On Tue, Jun 16, 2026 at 4:20 PM Holden Karau <[email protected]> >>>> wrote: >>>> >>>>> Also it's Data+AI summit so I suspect a lot of folks are busy with >>>>> that right now. >>>>> >>>>> On Tue, Jun 16, 2026 at 4:05 PM Wenchen Fan <[email protected]> >>>>> wrote: >>>>> >>>>>> No open bug report for 4.2 so far, but I found an issue in the API >>>>>> doc: https://github.com/apache/spark/pull/56551 . The generated >>>>>> proto java classes and the new internal Types Framework ops classes are >>>>>> leaked into the Scala API doc. >>>>>> >>>>>> Given we claim APIs in the API doc as public, I'm -1 to avoid >>>>>> releasing unexpected public APIs. >>>>>> >>>>>> On Mon, Jun 15, 2026 at 6:38 PM huaxin gao <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> The RC3 voting deadline has passed, but we have not received any >>>>>>> votes yet. >>>>>>> >>>>>>> Since the previous 72-hour voting period included a weekend, I’ll >>>>>>> keep the vote open and extend it for another 48 hours. Could community >>>>>>> members please help review RC3 and vote when you have a chance? Binding >>>>>>> votes from PMC members are especially needed to close the release vote. >>>>>>> >>>>>>> Also, SPARK-57452 >>>>>>> <https://issues.apache.org/jira/browse/SPARK-57452> was raised to >>>>>>> audit the 4.2 migration guide for potentially missing behavior-change >>>>>>> notes, and it is currently marked as a blocker. Could the owners of the >>>>>>> changes listed in SPARK-57452 please review the corresponding items and >>>>>>> confirm whether any of them should block RC3? If you are aware of any >>>>>>> other >>>>>>> 4.2 behavior changes that require migration-guide updates, please also >>>>>>> flag >>>>>>> them. >>>>>>> >>>>>>> If you believe SPARK-57452 should block the release, please vote -1 >>>>>>> and share the specific items that must be fixed before 4.2.0. Otherwise, >>>>>>> please help verify RC3 and vote. >>>>>>> >>>>>>> Thanks, >>>>>>> Huaxin >>>>>>> >>>>>>> On Fri, Jun 12, 2026 at 5:14 PM <[email protected]> wrote: >>>>>>> >>>>>>>> Please vote on releasing the following candidate as Apache Spark >>>>>>>> version 4.2.0. >>>>>>>> >>>>>>>> The vote is open until Mon, 15 Jun 2026 18:14:21 PDT and passes if >>>>>>>> a majority +1 PMC votes are cast, with >>>>>>>> a minimum of 3 +1 votes. >>>>>>>> >>>>>>>> [ ] +1 Release this package as Apache Spark 4.2.0 >>>>>>>> [ ] -1 Do not release this package because ... >>>>>>>> >>>>>>>> To learn more about Apache Spark, please see >>>>>>>> https://spark.apache.org/ >>>>>>>> >>>>>>>> The tag to be voted on is v4.2.0-rc3 (commit 560dc9d3c95): >>>>>>>> https://github.com/apache/spark/tree/v4.2.0-rc3 >>>>>>>> >>>>>>>> The release files, including signatures, digests, etc. can be found >>>>>>>> at: >>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc3-bin/ >>>>>>>> >>>>>>>> Signatures used for Spark RCs can be found in this file: >>>>>>>> https://downloads.apache.org/spark/KEYS >>>>>>>> >>>>>>>> The staging repository for this release can be found at: >>>>>>>> >>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1523/ >>>>>>>> >>>>>>>> The documentation corresponding to this release can be found at: >>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc3-docs/ >>>>>>>> >>>>>>>> The list of bug fixes going into 4.2.0 can be found at the >>>>>>>> following URL: >>>>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12356380 >>>>>>>> >>>>>>>> FAQ >>>>>>>> >>>>>>>> ========================= >>>>>>>> How can I help test this release? >>>>>>>> ========================= >>>>>>>> >>>>>>>> If you are a Spark user, you can help us test this release by taking >>>>>>>> an existing Spark workload and running on this release candidate, >>>>>>>> then >>>>>>>> reporting any regressions. >>>>>>>> >>>>>>>> If you're working in PySpark you can set up a virtual env and >>>>>>>> install >>>>>>>> the current RC via "pip install >>>>>>>> https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc3-bin/pyspark-4.2.0.tar.gz >>>>>>>> " >>>>>>>> and see if anything important breaks. >>>>>>>> In the Java/Scala, you can add the staging repository to your >>>>>>>> project's resolvers and test >>>>>>>> with the RC (make sure to clean up the artifact cache before/after >>>>>>>> so >>>>>>>> you don't end up building with an out of date RC going forward). >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> To unsubscribe e-mail: [email protected] >>>>>>>> >>>>>>>> >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Fight Health Insurance: https://www.fighthealthinsurance.com/ >>>>> <https://www.fighthealthinsurance.com/?q=hk_email> >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> Pronouns: she/her >>>>> >>>>
