felipepessoto opened a new issue, #12784:
URL: https://github.com/apache/gluten/issues/12784
### Backend
VL (Velox)
### Bug description
Gluten discards the **expression-level** `EvalMode` of `Cast` and instead
lets Velox decide cast failure/overflow behaviour from the **session-level**
`spark.sql.ansi.enabled`.
Consequently, a `Cast` whose own `evalMode` is `LEGACY` — which in Spark
must return `NULL` on failure — throws whenever the session flag happens to be
`true`.
This matters because Spark and Delta legitimately build such nodes:
`Cast(..., evalMode = LEGACY)` is created by `TableOutputResolver` (and by
Delta's `UpdateExpressionsSupport.castIfNeeded`) when
`spark.sql.storeAssignmentPolicy=LEGACY`, **independently** of
`spark.sql.ansi.enabled`. So a session can legitimately have
`spark.sql.ansi.enabled=true` while an individual `Cast` node is `LEGACY`.
### Repro
No Delta, no table, no write — static values only. Verified on Spark 3.5.5
with the Gluten Velox bundle.
```scala
import org.apache.spark.sql.Column
import org.apache.spark.sql.catalyst.expressions.{Cast, EvalMode}
import org.apache.spark.sql.functions.lit
import org.apache.spark.sql.types.IntegerType
// The session says ANSI ...
spark.conf.set("spark.sql.ansi.enabled", "true")
// ... but this Cast node explicitly says LEGACY, so Spark must return NULL.
val c = new Column(Cast(lit("abc").expr, IntegerType, None, EvalMode.LEGACY))
spark.range(1).select(c).show()
```
Session configuration used:
```
--conf spark.sql.ansi.enabled=true
--conf spark.gluten.sql.ansiFallback.enabled=false
--conf
spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding
```
Notes on those two extra confs:
* `spark.gluten.sql.ansiFallback.enabled=false` is exactly what Gluten's own
Delta test harness sets
(`backends-velox/src-delta40/test/scala/org/apache/spark/sql/delta/test/DeltaSQLCommandTest.scala`).
With the default `true` the query falls back to vanilla and the bug is masked.
* The excluded optimizer rules only prevent the literal from being
folded/evaluated on the driver, so the cast is actually executed by Velox. They
are not part of the bug.
### Actual behaviour
| Engine | Result |
| --- | --- |
| Vanilla Spark 3.5.5 | `null` |
| Gluten (Velox) | **throws** |
```
org.apache.gluten.exception.GlutenException: Exception: VeloxUserError
Error Source: USER
Reason: Cannot cast VARCHAR 'abc' to INTEGER. Invalid leading character: ""
```
### Expected behaviour
`null` on both engines. The `Cast` node's `evalMode` is `LEGACY`, so the
session-level ANSI flag must not apply to it.
### Which casts are affected today
Same harness, all with expression-level `evalMode = LEGACY` and session
`spark.sql.ansi.enabled=true`. Every row should be a non-throwing result:
| Cast | Vanilla | Gluten |
| --- | --- | --- |
| `string 'abc'` → `int` | `null` | **throws** |
| `string '9999999999'` → `int` (overflow) | `null` | **throws** |
| `string 'xyz'` → `date` | `null` | **throws** |
| `decimal(4,1)` → `decimal(3,2)` (overflow) | `null` | `null` (see below) |
| `decimal(4,1)` → `tinyint` (overflow) | `-25` | `-25` |
| `bigint` → `int` (overflow) | `1410065407` | `1410065407` |
| `double` → `int` (overflow) | `2147483647` | `2147483647` |
The rows that currently agree do so only because Velox does not (yet)
enforce ANSI for those particular conversions — not because Gluten handles
`evalMode` correctly. As Velox adds ANSI coverage, more rows flip to "throws".
That is precisely what just happened for `decimal → decimal`, see *Impact*
below.
### Root cause
1.
`gluten-substrait/src/main/scala/org/apache/gluten/expression/UnaryExpressionTransformer.scala`
(`CastTransformer`) forwards only whether the mode is `TRY`:
```scala
ExpressionBuilder.makeCast(
typeNode,
child.doTransform(context),
SparkShimLoader.getSparkShims.withTryEvalMode(original))
```
`EvalMode.LEGACY` and `EvalMode.ANSI` therefore produce **identical**
Substrait nodes — the distinction is lost before it ever reaches native code.
2.
`gluten-substrait/src/main/scala/org/apache/gluten/config/GlutenConfig.scala`
copies the session-level flag into the native conf map:
```scala
(SQLConf.ANSI_ENABLED.key, SQLConf.ANSI_ENABLED.defaultValueString)
```
3. `cpp/core/config/GlutenConfig.h` reads it natively:
```cpp
const std::string kAnsiEnabled = "spark.sql.ansi.enabled";
```
So Velox applies one query-wide ANSI setting to **every** cast, rather than
honouring each `Cast`'s own `evalMode`.
This is already acknowledged in Gluten's own test code —
`gluten-ut/spark40/src/test/scala/org/apache/spark/sql/catalyst/expressions/GlutenCastWithAnsiOnSuite.scala`:
> `CastWithAnsiOnSuite` creates `Cast` expressions with `EvalMode.ANSI` but
does not set the session-level ANSI config. Velox reads ANSI mode from session
config to decide cast behavior [...]. We must sync session config with the
expression-level `evalMode` [...]
### Impact
This is the root cause of the Delta CI regressions tracked in #12779.
Delta's `ImplicitMergeCastingSuite` / `TypeWideningInsertSchemaEvolution*`
tests run with `spark.sql.storeAssignmentPolicy=LEGACY` and
`spark.sql.ansi.enabled=true`, so Delta emits `Cast(evalMode = LEGACY)` and the
tests expect **no exception**. Gluten reports ANSI to Velox and Velox throws.
Those tests only started failing after the `2026_08_14` daily Velox bump
(#12769), which pulled in upstream Velox `a96d0415c` *"Enable ANSI-compliant
cast from DECIMAL to DECIMAL"*. Before that bump Velox simply had no ANSI
enforcement for `decimal → decimal`, which masked the defect for that
conversion — on a pre-bump build even a genuine `EvalMode.ANSI` `decimal →
decimal` cast returns `null`. The Velox change is correct; it merely exposed
the pre-existing `evalMode` handling gap in Gluten.
### Suggested fix
Propagate the per-expression `EvalMode` (`LEGACY` / `ANSI` / `TRY`) through
Substrait to Velox and evaluate each cast accordingly, instead of relying on
the session-wide `spark.sql.ansi.enabled`.
### Spark version
Reproduced on Spark 3.5.5 (Gluten `1.8.0-SNAPSHOT`). The mechanism is
version independent and applies to Spark 3.4+ where `Cast` carries `evalMode`.
### Spark configurations
```
spark.sql.ansi.enabled=true
spark.gluten.sql.ansiFallback.enabled=false
```
### Are you willing to submit a PR?
- [ ] Yes, I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
