luis4a0 commented on code in PR #13282:
URL: https://github.com/apache/gluten/pull/13282#discussion_r4583709120
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxValidatorApi.scala:
##########
@@ -39,17 +40,36 @@ import io.substrait.proto.SimpleExtensionDeclaration
import scala.collection.JavaConverters._
import scala.collection.mutable.ArrayBuffer
-class VeloxValidatorApi extends ValidatorApi {
+class VeloxValidatorApi extends ValidatorApi with Logging {
import VeloxValidatorApi._
- /** For velox backend, key validation is on native side. */
- override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = {
- expr match {
- // Both format_string and printf resolve to Spark's FormatString
expression.
- case formatString: FormatString =>
- supportsFormatString(formatString)
- case _ => true
- }
+ /**
+ * Enforce integral ROUND eligibility on the JVM to preserve Spark's
captured ANSI semantics,
+ * independently of optional native plan validation.
+ */
+ override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = expr match {
+ // Both format_string and printf resolve to Spark's FormatString
expression.
+ case formatString: FormatString =>
+ supportsFormatString(formatString)
+ case round: Round =>
+ round.child.dataType match {
+ case ByteType | ShortType | IntegerType | LongType =>
+ // Negative scales can overflow, and Spark captures ANSI mode in the
expression.
+ val supported = round.scale match {
+ case Literal(null, IntegerType) => true
+ case Literal(scale: Int, IntegerType) =>
+ scale >= 0 && scale <= MaxQualifiedIntegralRoundScale
+ case _ => false
+ }
+ if (!supported) {
+ logDebug(
+ "Integral ROUND requires a folded INTEGER scale in " +
+ s"[0, $MaxQualifiedIntegralRoundScale] or NULL; retaining
Spark.")
+ }
+ supported
+ case _ => true
+ }
+ case _ => true
}
Review Comment:
Corrected the PR description at https://github.com/apache/gluten/pull/13282.
It now states that negative, out-of-range, and nonliteral integral scales
always remain on Spark independently of optional native plan validation,
matching the unconditional JVM `doExprValidate` guard and implementation
comment.
##########
cpp/velox/tests/SparkFunctionTest.cc:
##########
@@ -125,6 +126,114 @@ TEST_F(SparkFunctionTest, roundWithDecimal) {
runRoundWithDecimalTest<int8_t>(testRoundWithDecIntegralData<int8_t>());
}
+TEST_F(SparkFunctionTest, integralRoundUsesSessionAnsi) {
+ const auto input = makeRowVector({makeNullableFlatVector<int8_t>({127, -128,
25, -25, std::nullopt})});
+ for (const auto ansi : {false, true}) {
+ queryCtx_->testingOverrideConfigUnsafe({{sparkAnsiEnabledConfigKey(), ansi
? "true" : "false"}});
+ const auto expression = "round(c0, cast(-1 as integer))";
+ if (ansi) {
+ VELOX_ASSERT_USER_THROW(evaluate(expression, input), "Arithmetic
overflow: round(");
Review Comment:
I retained this focused assertion intentionally. Gluten `AGENTS.md` requires
tests to assert informative error-message text rather than only exception types
or categories. The prefix `Arithmetic overflow: round(` checks the stable
actionable diagnosis without binding dynamic argument values; the native
function binary passes 25/25.
##########
docs/backends/velox/velox-backend-scalar-function-support.md:
##########
@@ -297,7 +297,7 @@ The status applies to `spark.sql.ansi.enabled=false`. When
ANSI mode is enabled,
| randn | Randn | S | |
| random | Rand | S | |
| rint | Rint | S | |
-| round | Round | S | |
+| round | Round | PS | Integral scales in
[0, 400] and NULL use upstream Velox. NULL scales skip child evaluation.
Negative scales, scales outside [0, 400], and scales not folded to INTEGER
literals fall back to Spark to preserve captured ANSI behavior. |
Review Comment:
Addressed in signed commit
https://github.com/luis4a0/gluten/commit/0aa564099. The table now states the
native eligibility range, Spark fallback, and NULL short-circuit contract
concisely. `tools/scripts/diff-function-support-docs.py` passes with unchanged
support counts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]