LuciferYang opened a new pull request, #13004:
URL: https://github.com/apache/gluten/pull/13004
## What changes are proposed in this pull request?
`testWithMinSparkVersion(name, floor)` registers a case only when the
running Spark is at least `floor` (`GlutenQueryTest.scala:132`). Since #12902
the supported set is {3.4, 3.5, 4.0, 4.1}, so any floor of 3.4 or below is
always satisfied and the call is a plain `test(...)` wearing a version check.
This rewrites all 95 of them across 23 files: 84 with floor `"3.4"`, 9 with
`"3.3"`, 2 with `"3.2"`.
#12981 left the 3.4 group out of scope deliberately. The 3.3 and 3.2 groups
are newer than that cleanup rather than older: #12454 branched before #12954
and merged between #12954 and #12981, so it brought the pattern back into
`backends-bolt` after the first sweep and before the second one, which touched
no bolt file.
The helper stays. Ten call sites still pass `"3.5"`, and that floor does
separate 3.4 from the rest.
Three things came off with the floors rather than being left dangling.
`BoltHashJoinSuite.scala` had a `"3.2"` floor whose body still branched on
`startsWith("3.2.")` in two places, so the floor and the assertions were one
thing and removing only the floor would have been half a job; both `3.2.` arms
are gone and the surviving expectations are the ones the supported versions
already take. Two comments existed only to justify a floor and now say
something untrue or pointless: `ScalarFunctionsValidateSuite.scala:655` claimed
`We currently test below functions with Spark v3.4` above a group that
registers on all four, and `DeltaSuite.scala:843` noted that TIMESTAMP_NTZ
arrived in Spark 3.4, which constrains nothing once 3.4 is the minimum.
Otherwise no test body changed. The rewrite was done by script and then run
through `spotless:apply`, so the remaining edits are the call itself and
whatever reflowing scalafmt wanted afterwards; the collapsed multi-line calls
are where the line counts move.
## How was this patch tested?
`clean test-compile` on Spark 3.4, 3.5, 4.0 and 4.1 with `-Pbackends-velox
-Pspark-ut -Piceberg -Pdelta`, Scala 2.13, which covers the velox,
gluten-delta, gluten-iceberg and gluten-ut files. The 3.5 run adds
`-Pbackends-clickhouse` for `src-iceberg/test`, that being the only profile
clickhouse builds on.
The nine `backends-bolt` files needed a detour, since bolt does not compile
on main at all right now. I stacked #12999 on top of this branch, ran
`test-compile -Pbackends-bolt -Pspark-ut` on 3.4 and 3.5, then dropped it
again, so what is pushed here is the rewrite alone.
No suite was run. For the 95 rewritten calls the change cannot alter what a
case does, only whether it is registered, and every one of them was already
registered on all four supported versions; a script checked that the set of
case names each of the 23 files registers is identical before and after. The
`BoltHashJoinSuite` assertion change is the one place that needed judgement
rather than mechanics, and it cannot be run here: bolt's native library needs
Linux plus GCC 10-12 or Clang 16 per `docs/bolt-quick-start.md`.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-5
Related issue: #13003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
