luis4a0 commented on code in PR #13103:
URL: https://github.com/apache/gluten/pull/13103#discussion_r4283756654
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxValidatorApi.scala:
##########
@@ -38,12 +39,26 @@ import io.substrait.proto.SimpleExtensionDeclaration
import scala.collection.JavaConverters._
import scala.collection.mutable.ArrayBuffer
-class VeloxValidatorApi extends ValidatorApi {
+class VeloxValidatorApi extends ValidatorApi with Logging {
import VeloxValidatorApi._
- /** For velox backend, key validation is on native side. */
- override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean =
- true
+ /**
+ * Enforce decimal ROUND scale compatibility on the JVM independently of
optional native plan
+ * validation.
+ */
+ override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = expr match {
+ case round: Round if round.child.dataType.isInstanceOf[DecimalType] =>
+ val supported = round.scale match {
+ case Literal(null, IntegerType) => true
+ case Literal(scale: Int, IntegerType) => scale >= -400 && scale <= 400
+ case _ => false
+ }
+ if (!supported) {
+ logDebug("Decimal ROUND requires a folded INTEGER scale in [-400, 400]
or NULL.")
+ }
+ supported
+ case _ => true
+ }
Review Comment:
Addressed in
https://github.com/apache/gluten/commit/80a101980bc5b20dcc13bc84d4a73187fbd3b3a9
. The guard now requires a foldable INTEGER expression, evaluates it, and
checks the same NULL/[-400, 400] domain. Wrong types and
non-foldable/out-of-range inputs remain rejected; evaluation errors are not
swallowed. The existing DecimalRoundTransformer already emits the evaluated
scale as a typed literal, so native validation still receives its required
constant representation.
Added API and SQL regressions with ConstantFolding and NullPropagation
disabled. The SQL test explicitly verifies that `1 - 2` remains non-Literal in
the optimized plan, then checks native execution and Spark-equivalent
results/schema, plus out-of-range fallback, with native validation on and off.
The updated suites passed on Spark 3.4.4, 3.5.5, 4.0.2 and 4.1.1; Spark 4.1.1
also passed on JDK 21.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
