LuciferYang opened a new issue, #12885:
URL: https://github.com/apache/gluten/issues/12885
### What
`VeloxTestSettings` excludes six cases from `GlutenTryCastSuite` on Spark
3.4, 3.5, 4.0 and 4.1. All six pass when the exclusions are removed, so the
exclusions cost coverage without buying anything.
```scala
enableSuite[GlutenTryCastSuite]
...
.exclude("ANSI mode: Throw exception on casting out-of-range value to byte
type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to
short type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to int
type")
.exclude("ANSI mode: Throw exception on casting out-of-range value to long
type")
.exclude("cast from invalid string to numeric should throw
NumberFormatException")
.exclude("SPARK-26218: Fix the corner case of codegen when casting float
to Integer")
```
The same six entries appear in all four modules. Unlike the neighbouring
exclusions in that block, none of them carries a comment explaining what fails.
### Evidence
Removing the six from `gluten-ut/spark34` and running the suite locally
against Spark 3.4 takes it from 72 to 78 cases with no new failures. The six
show up as passed. The run also reports two failures in `Gluten - data type
casting` and `Gluten - cast string to timestamp`, both of which are
pre-existing timezone cases unrelated to this change and green in CI, so a
local run is not authoritative on its own. That is why the accompanying PR
opens as a draft: CI on all four versions is the check that matters here.
### Why they are probably there
On Spark 3.3 these cases live in `AnsiCastSuiteBase`, and `TryCastSuite`
builds them through `TryCast`, a 3.3-only expression class. Gluten's expression
mapping is keyed by class (`Sig(expClass: Class[_], name: String)`) and has no
`Sig[TryCast]`, so on 3.3 a plan containing `TryCast` cannot be offloaded and
falls back to vanilla Spark. The suite's log confirms it: `Skipping evaluation
- Nonempty inputRow cannot be converted to DataFrame` appears ten times in a
3.3 run. Gluten's own try_cast validation is gated the same way,
`testWithMinSparkVersion("try_cast", "3.4")` in `ScalarFunctionsValidateSuite`.
Spark 3.4 replaced `TryCast` with `Cast(child, dataType, timeZoneId,
EvalMode.TRY)`, which Gluten does offload (`Spark34Shims.withTryEvalMode`,
`UnaryExpressionTransformer`, `VeloxSparkPlanExecApi`). So the cases went from
never touching Velox on 3.3 to genuinely exercising it on 3.4+, which is a
plausible reason for someone to have excluded them while bringing the newer
modules up. Whatever failed at that point passes now.
### Context
Found while inventorying which vanilla cases run on `gluten-ut/spark33` but
are excluded on every surviving version, as a prerequisite for removing that
module under #12807. Tracked in #12863. That inventory turned up 17 candidates;
these six are the only ones that pass once enabled. The rest either fail
locally in a way consistent with the reasons written next to them, or are
ClickHouse-only and cannot be checked without a CH build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
