dongjoon-hyun opened a new pull request, #993: URL: https://github.com/apache/spark-kubernetes-operator/pull/993
### What changes were proposed in this pull request? This PR aims to upgrade docker images to Apache Spark 4.3.0, 4.1.4, and 4.0.5. - [v4.3.0](https://github.com/apache/spark/releases/tag/v4.3.0) (2026-10-27) - [v4.1.4](https://github.com/apache/spark/releases/tag/v4.1.4) (2026-10-21) - [v4.0.5](https://github.com/apache/spark/releases/tag/v4.0.5) (2026-10-21) Specifically, - Update all examples, E2E tests, benchmark script, and docs from `4.2.0` to `4.3.0`. - Update the default Spark cluster image of `SparkClusterResourceSpec` from `apache/spark:4.2.0` to `apache/spark:4.3.0`. - Update `pi-with-comet.yaml` and `spark-connect-server-iceberg.yaml` from `4.1.3` to `4.1.4`, and `pi-with-gluten.yaml` from `4.0.4` to `4.0.5`. They stay on the Spark 4.1 and 4.0 lines because their `Apache DataFusion Comet`, `Apache Iceberg`, and `Apache Gluten` artifacts are built for them. ### Why are the changes needed? Since the `apache/spark` docker images of these releases are published, the examples and tests should use them. The build dependency is already on Spark 4.3.0 since SPARK-60793, which did not change the images because they were not published at that time. ### Does this PR introduce _any_ user-facing change? Yes, the default Spark cluster image becomes `apache/spark:4.3.0` when `spark.kubernetes.container.image` is not set. `docs/migration_guide.md` describes it and how to restore the previous behavior. The other changes are limited to docs, examples, and tests. ### How was this patch tested? Pass the CIs. I also verified that all referenced image tags exist in Docker Hub, including the ones resolved from `apache/spark:{{SPARK_VERSION}}*` with `spec.runtimeVersions.sparkVersion`: - `4.3.0`, `4.3.0-scala`, `4.3.0-scala-java17`, `4.3.0-scala-java25`, `4.3.0-python3` - `4.1.4`, `4.1.4-scala` - `4.0.5-scala` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
