LuciferYang opened a new issue, #13088: URL: https://github.com/apache/gluten/issues/13088
### Description After the Spark 3.3 removal (#12807) and the follow-up cleanup (#13003), two Spark 3.2/3.3 references remain that I did not want to change unilaterally, because each needs a decision from the code's owner rather than a mechanical edit. ### 1. `Makefile`: dead `-Pspark-3.3` / `-Pspark-3.2` / `-Pspark-3.2-las` targets The bolt `Makefile` (added with #12454, not wired into any CI workflow) still drives several targets off profiles that no longer exist: - `jar` (`:158`) builds with `-Pspark-3.3` - `jar-skip-check` (`:164`), `test` (`:211`), `clean` (`:224`) build with `-Pspark-3.2` - `spark32-las` (`:169-170`) builds with `-Pspark-3.2-las` - `jar_spark33` (`:192-196`) builds with `-Pspark-3.3`; `jar_spark33` is also in `.PHONY` (`:83`) - `fast-jar` / `zip` / `fast-zip` and the `jar`/`jar-skip-check`/`spark32-las` output names are hardcoded `gluten-spark3.2_2.12-...` (already inconsistent, since `jar` builds 3.3 but names the artifact 3.2) Because Maven only warns on an unknown `-P`, these fall through to the root-pom default properties (`sparkbundle.version=3.5` / `sparkshim=spark35` but `spark.version=4.0.2`) with `-Denforcer.skip=true` suppressing the guard, so `make jar` today produces a nonsense build rather than an error. `jar_spark34` / `jar_spark35` / `test_spark35` are the working targets. The fix is presumably to repoint the defaults to a supported version (3.5 matches the repo default and bolt builds on it) and drop the 3.2/3.3-only targets, but the default target and the output-naming scheme are bolt's call, so I am leaving it here rather than guessing. cc the bolt owners. ### 2. `cpp-ch/local-engine/tests/json/native_write_plan_1_spark33.json` This fixture, and its sibling `native_write_plan_1_spark35.json`, appear unreferenced: the only `INCBIN` loads in `gtest_write_pipeline.cpp` are `native_write_plan.json` and `native_write_one_partition.json`, and a tree-wide grep for `native_write_plan_1_spark` finds no loader. If they are genuinely dead, the `spark33` one is a 3.3 leftover and the `spark35` one is orphaned too. I cannot build or run the ClickHouse tests locally, so I would rather a CH maintainer confirm these are unused before they are deleted. cc the ClickHouse owners. ### Handled separately The `tools/workload/benchmark_velox/initialize.ipynb` reference to `spark-3.3.1` (it provisioned a Spark that Gluten can no longer run on) is a straightforward version bump and is fixed in the PR that references this issue, together with a stale `Spark 3.3/3.4` comment in `WriteFilesExecTransformer`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
