yaooqinn commented on PR #12177:
URL: https://github.com/apache/gluten/pull/12177#issuecomment-4569891851
CI root cause identified — pushed a fix-up commit (`Upgrade Delta to 4.2.0
for Spark 4.1`).
**Diagnosis:**
`spark-test-spark41` / `spark-test-spark41-slow` failed with:
```
*** RUN ABORTED ***
java.lang.NoClassDefFoundError:
org/apache/spark/sql/catalyst/plans/logical/IgnoreCachedData
at
org.apache.spark.sql.delta.catalog.AbstractDeltaCatalog.$anonfun$alterTable$9(AbstractDeltaCatalog.scala:952)
Cause: java.lang.ClassNotFoundException:
org.apache.spark.sql.catalyst.plans.logical.IgnoreCachedData
```
Spark 4.1.2 **removed** the public trait
`org.apache.spark.sql.catalyst.plans.logical.IgnoreCachedData` — the file
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/IgnoreCachedData.scala`
is `removed` in the `v4.1.1...v4.1.2` diff, and all internal `with
IgnoreCachedData` mixins were stripped (`ResetCommand`, `ClearCacheCommand`,
`CreateUserDefinedFunctionCommand`, etc.).
Delta Lake 4.1.0 was compiled against 4.1.1 and its
`AbstractDeltaCatalog.alterTable` still references the trait at the bytecode
level → `NoClassDefFoundError` on Spark 4.1.2.
Gluten itself has zero references to `IgnoreCachedData`.
**Fix:**
Bumped `<delta.version>` from `4.1.0` to `4.2.0` in the `spark-4.1` profile.
Verified the 4.2.0 jar no longer references `IgnoreCachedData`. CI will tell us
if Delta 4.2.0 internal API changes break anything in `src-delta40/`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
