luis4a0 commented on code in PR #13282:
URL: https://github.com/apache/gluten/pull/13282#discussion_r4591776341
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxValidatorApi.scala:
##########
@@ -39,17 +40,36 @@ import io.substrait.proto.SimpleExtensionDeclaration
import scala.collection.JavaConverters._
import scala.collection.mutable.ArrayBuffer
-class VeloxValidatorApi extends ValidatorApi {
+class VeloxValidatorApi extends ValidatorApi with Logging {
import VeloxValidatorApi._
- /** For velox backend, key validation is on native side. */
- override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = {
- expr match {
- // Both format_string and printf resolve to Spark's FormatString
expression.
- case formatString: FormatString =>
- supportsFormatString(formatString)
- case _ => true
- }
+ /**
+ * Enforce integral ROUND eligibility on the JVM to preserve Spark's
captured ANSI semantics,
+ * independently of optional native plan validation.
+ */
+ override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = expr match {
Review Comment:
The current PR description at https://github.com/apache/gluten/pull/13282
already states that negative, out-of-range, and nonliteral integral scales
always remain on Spark independently of optional native plan validation. This
matches the unconditional JVM `doExprValidate` guard.
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxValidatorApi.scala:
##########
@@ -39,17 +40,36 @@ import io.substrait.proto.SimpleExtensionDeclaration
import scala.collection.JavaConverters._
import scala.collection.mutable.ArrayBuffer
-class VeloxValidatorApi extends ValidatorApi {
+class VeloxValidatorApi extends ValidatorApi with Logging {
import VeloxValidatorApi._
- /** For velox backend, key validation is on native side. */
- override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = {
- expr match {
- // Both format_string and printf resolve to Spark's FormatString
expression.
- case formatString: FormatString =>
- supportsFormatString(formatString)
- case _ => true
- }
+ /**
+ * Enforce integral ROUND eligibility on the JVM to preserve Spark's
captured ANSI semantics,
+ * independently of optional native plan validation.
+ */
+ override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = expr match {
+ // Both format_string and printf resolve to Spark's FormatString
expression.
+ case formatString: FormatString =>
+ supportsFormatString(formatString)
+ case round: Round =>
+ round.child.dataType match {
+ case ByteType | ShortType | IntegerType | LongType =>
+ // Negative scales can overflow, and Spark captures ANSI mode in the
expression.
+ val supported = round.scale match {
+ case Literal(null, IntegerType) => true
+ case Literal(scale: Int, IntegerType) =>
+ scale >= 0 && scale <= MaxQualifiedIntegralRoundScale
+ case _ => false
+ }
Review Comment:
The current PR description at https://github.com/apache/gluten/pull/13282
already states that negative, out-of-range, and nonliteral integral scales
always remain on Spark independently of optional native plan validation. This
matches the unconditional JVM `doExprValidate` guard.
##########
cpp/velox/substrait/SubstraitToVeloxPlanValidator.cc:
##########
@@ -141,17 +141,30 @@ bool
SubstraitToVeloxPlanValidator::flattenDualLevel(const TypePtr& type, std::v
bool SubstraitToVeloxPlanValidator::validateRound(
const ::substrait::Expression::ScalarFunction& scalarFunction,
- const RowTypePtr& inputType) {
+ const std::vector<core::TypedExprPtr>& params) {
const auto& arguments = scalarFunction.arguments();
if (arguments.size() < 2) {
return false;
}
+ if (params.empty() || !params[0]) {
+ LOG_VALIDATION_MSG("Round value expression is unavailable.");
+ return false;
+ }
if (!arguments[1].value().has_literal()) {
- LOG_VALIDATION_MSG("Round scale is expected.");
+ LOG_VALIDATION_MSG("Round scale must be a literal integer for native
execution.");
return false;
}
+ const auto& scale = arguments[1].value().literal();
+ if (scale.has_null() && scale.null().has_i32()) {
+ const auto& valueType = params[0]->type();
+ if (!valueType->isDecimal() &&
+ (valueType->isTinyint() || valueType->isSmallint() ||
valueType->isInteger() || valueType->isBigint())) {
+ return true;
+ }
+ }
+
// Velox has different result with Spark on negative scale.
auto typeCase = arguments[1].value().literal().literal_type_case();
switch (typeCase) {
Review Comment:
Addressed in signed head
https://github.com/luis4a0/gluten/commit/cdbfc377b2416f5392d1477aab3c3e5929864864.
Native validation now applies the `[0, 400]` upper bound only to exact
primitive integral inputs, matching the JVM gate while preserving
floating/decimal behavior. Tests accept 400, reject 401 and INTEGER_MAX, and
verify the fallback diagnostic; native tests pass 25/25 and Scala validation
passes 2/2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]