Can we include the log4j 2.26.1 upgrading in next RC? TL:DR, the current log4j2 2.25.x used by branch-4.2 may throw NPE on logging, downgrading to 2.24.x can fix the issue but bring some CVEs back, log4j2 2.26.1 is in the voting phase and likely available soon. See full context at [1]
[1] https://github.com/apache/spark/pull/51719#issuecomment-3341344974 Thanks, Cheng Pan > On Jun 30, 2026, at 08:52, huaxin gao <[email protected]> wrote: > > Thanks Szehon for reporting the problem. I'm failing RC4, and will roll RC5 > once the fix <https://github.com/apache/spark/pull/56875> is merged and > backported to branch-4.2. > > > Thanks to everyone who tested RC4! > > Huaxin > > > On Mon, Jun 29, 2026 at 5:14 PM Szehon Ho <[email protected] > <mailto:[email protected]>> wrote: >> I made a fix for this, it would be great to get it into the release. >> >> https://github.com/apache/spark/pull/56875 >> >> Thanks Huaxin! >> Szehon >> >> >> On Mon, Jun 29, 2026 at 4:54 PM Szehon Ho <[email protected] >> <mailto:[email protected]>> wrote: >>> Sorry I was informed of a potential vulnerability/problem in new fromWKB in >>> Spark 4.2, looking now at it. >>> >>> Thanks >>> Szehon >>> >>> On Mon, Jun 29, 2026 at 1:05 PM Allison Wang <[email protected] >>> <mailto:[email protected]>> wrote: >>>> +1 >>>> >>>> On Mon, Jun 29, 2026 at 10:34 AM Max Gekk <[email protected] >>>> <mailto:[email protected]>> wrote: >>>>> +1 >>>>> >>>>> On Mon, Jun 29, 2026 at 6:32 PM Xiao Li <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> > >>>>> > +1 (binding) >>>>> > >>>>> > Verified on Linux (Ubuntu, JDK 17, Python 3.11): >>>>> > >>>>> > - Signatures: every artifact (source, the three binary distributions, >>>>> > the pyspark/pyspark_client/pyspark_connect tarballs, and SparkR) has a >>>>> > good GPG signature from the RM key >>>>> > 709226B910E0F10917123B6259B586ADA5A538D1, which is in KEYS. >>>>> > - Checksums: SHA512 matches for all artifacts.- Tag/commit: v4.2.0-rc4 >>>>> > resolves to f92a807c06b, and the RELEASE metadata embedded in the >>>>> > binary distributions records the same git revision. >>>>> > - Source build: compiled and packaged spark-core and its module >>>>> > dependencies from the source tarball with ./build/mvn on JDK 17 (BUILD >>>>> > SUCCESS). >>>>> > - Binary distribution: ran SparkPi and a spark-shell Scala job (range >>>>> > aggregation + Spark SQL) — results correct. >>>>> > - PySpark: pip-installed pyspark-4.2.0.tar.gz and ran DataFrame, Spark >>>>> > SQL, and a Python UDF in local mode — results correct. >>>>> > >>>>> > Thanks Huaxin for driving the release! >>>>> > >>>>> > Xiao >>>>> > >>>>> > Uroš Bojanić <[email protected] <mailto:[email protected]>> 于2026年6月29日周一 >>>>> > 09:12写道： >>>>> >> >>>>> >> +1 (non-binding) >>>>> >> >>>>> >> verified RC4 (tag v4.2.0-rc4, commit f92a807c06b) on macOS/arm64. >>>>> >> >>>>> >> - Good GPG signatures and SHA512 sums on all artifacts (source, three >>>>> >> binaries, the PySpark/SparkR tarballs) against KEYS. >>>>> >> - Built the full source tree from the tag with -Phive >>>>> >> -Phive-thriftserver, got a clean BUILD SUCCESS across all 39 modules. >>>>> >> - Ran quick smoke tests across every API surface: Java, Scala, SQL, >>>>> >> and PySpark job from the bundled distro; all look good. >>>>> >> - Sanity-checked the binary dist (RELEASE, LICENSE/NOTICE/licenses all >>>>> >> present) and ran dev/check-license; RAT passes. >>>>> >> - Diffed the docs against 4.1.0 and analyzed the changes (new pages, >>>>> >> migration guides and version refs); all look good. >>>>> >> >>>>> >> Thank you Huaxin Gao! >>>>> >> >>>>> >> On 2026/06/27 00:21:32 [email protected] >>>>> >> <mailto:[email protected]> wrote: >>>>> >> > Please vote on releasing the following candidate as Apache Spark >>>>> >> > version 4.2.0. >>>>> >> > >>>>> >> > The vote is open until Mon, 29 Jun 2026 18:21:31 PDT and passes if a >>>>> >> > majority +1 PMC votes are cast, with >>>>> >> > a minimum of 3 +1 votes. >>>>> >> > >>>>> >> > [ ] +1 Release this package as Apache Spark 4.2.0 >>>>> >> > [ ] -1 Do not release this package because ... >>>>> >> > >>>>> >> > To learn more about Apache Spark, please see >>>>> >> > https://spark.apache.org/ >>>>> >> > >>>>> >> > The tag to be voted on is v4.2.0-rc4 (commit f92a807c06b): >>>>> >> > https://github.com/apache/spark/tree/v4.2.0-rc4 >>>>> >> > >>>>> >> > The release files, including signatures, digests, etc. can be found >>>>> >> > at: >>>>> >> > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-bin/ >>>>> >> > >>>>> >> > Signatures used for Spark RCs can be found in this file: >>>>> >> > https://downloads.apache.org/spark/KEYS >>>>> >> > >>>>> >> > The staging repository for this release can be found at: >>>>> >> > https://repository.apache.org/content/repositories/orgapachespark-1524/ >>>>> >> > >>>>> >> > The documentation corresponding to this release can be found at: >>>>> >> > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-docs/ >>>>> >> > >>>>> >> > The list of bug fixes going into 4.2.0 can be found at the following >>>>> >> > URL: >>>>> >> > https://issues.apache.org/jira/projects/SPARK/versions/12356380 >>>>> >> > >>>>> >> > FAQ >>>>> >> > >>>>> >> > ========================= >>>>> >> > How can I help test this release? >>>>> >> > ========================= >>>>> >> > >>>>> >> > If you are a Spark user, you can help us test this release by taking >>>>> >> > an existing Spark workload and running on this release candidate, >>>>> >> > then >>>>> >> > reporting any regressions. >>>>> >> > >>>>> >> > If you're working in PySpark you can set up a virtual env and install >>>>> >> > the current RC via "pip install >>>>> >> > https://dist.apache.org/repos/dist/dev/spark/v4.2.0-rc4-bin/pyspark-4.2.0.tar.gz"; >>>>> >> > and see if anything important breaks. >>>>> >> > In the Java/Scala, you can add the staging repository to your >>>>> >> > project's resolvers and test >>>>> >> > with the RC (make sure to clean up the artifact cache before/after so >>>>> >> > you don't end up building with an out of date RC going forward). >>>>> >> > >>>>> >> > --------------------------------------------------------------------- >>>>> >> > To unsubscribe e-mail: [email protected] >>>>> >> > <mailto:[email protected]> >>>>> >> > >>>>> >> > >>>>> >> >>>>> >> --------------------------------------------------------------------- >>>>> >> To unsubscribe e-mail: [email protected] >>>>> >> <mailto:[email protected]> >>>>> >> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe e-mail: [email protected] >>>>> <mailto:[email protected]> >>>>>
