brijrajk commented on code in PR #12199:
URL: https://github.com/apache/gluten/pull/12199#discussion_r3331660371
##########
backends-velox/src/test/scala/org/apache/gluten/functions/MathFunctionsValidateSuite.scala:
##########
@@ -66,7 +66,15 @@ class MathFunctionsValidateSuiteAnsiOn extends
FunctionsValidateSuite {
}
}
-abstract class MathFunctionsValidateSuite extends FunctionsValidateSuite {
+class MathFunctionsValidateSuite extends FunctionsValidateSuite {
+
+ // Disable ANSI mode: Spark 4 enables it by default, which wraps math
functions
+ // in ANSI check nodes and prevents ProjectExecTransformer from being the
top-level
+ // plan node. ANSI-specific behaviour is tested in
MathFunctionsValidateSuiteAnsiOn.
+ override protected def sparkConf: SparkConf = {
+ super.sparkConf
+ .set(SQLConf.ANSI_ENABLED.key, "false")
Review Comment:
Thanks for digging into this. I investigated the full flow:
`SPARK_ANSI_SQL_MODE: false` is set as a GitHub Actions env variable in
`velox_backend_x86.yml` but it never reaches Maven tests — the Spark 4.0 test
jobs pass only `-Dspark.test.home=...` in `DargLine`, and the
`scalatest-maven-plugin` config in `pom.xml` only forwards
`log4j.configurationFile` as a system property. No Scala test code in the
`sparkConf` chain reads the env var either.
So the env var is effectively documentation-only today, and
`spark.sql.ansi.enabled` can only be controlled via the `sparkConf` override in
test code.
The right fix would be to wire `SPARK_ANSI_SQL_MODE` into
`FunctionsValidateSuite.sparkConf`:
```scala
val ansiEnabled = sys.env.getOrElse("SPARK_ANSI_SQL_MODE", "false")
super.sparkConf.set(SQLConf.ANSI_ENABLED.key, ansiEnabled)
```
This propagates to all subclasses (no per-suite overrides needed), while
`MathFunctionsValidateSuiteAnsiOn` can still override it back to `true` for
ANSI-specific tests.
Would it make sense to raise a separate issue for this infrastructure fix?
Happy to take it on as a follow-up. For this PR, should I keep the per-suite
override as a self-contained fix, or would you prefer I address the root cause
here directly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
