mixermt commented on PR #5898: URL: https://github.com/apache/datafusion-comet/pull/5898#issuecomment-6764920581
@andygrove The merge queue failures are a real bug in this PR, not a flake. **Cause** - On the Spark 4.0 profile (the macOS scans job), the encryption-zone test in `CometIcebergHdfsSuite` aborts the run with `NoClassDefFoundError: org/bouncycastle/jce/provider/BouncyCastleProvider`. - Spark 4.0 uses Hadoop 3.4.1, whose `KeyProvider` references BouncyCastle directly, and BouncyCastle is not on Comet's test classpath. - Hadoop 3.4.2 (Spark 4.1) loads it via reflection, and Hadoop 3.3.4 (Spark 3.5) does not reference it. That is why the PR checks passed. **Fix** - Add `org.bouncycastle:bcprov-jdk18on` as a test-scoped dependency in `spark/pom.xml`, and correct the suite's Scaladoc. - I'll verify the suite locally on Spark 4.0 and 4.1, push, and ping you to re-queue. The Scala lint failure in the last run was a Maven Central HTTP 429 and is unrelated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
