Copilot commented on code in PR #13282:
URL: https://github.com/apache/gluten/pull/13282#discussion_r4583685812
##########
backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxValidatorApi.scala:
##########
@@ -39,17 +40,36 @@ import io.substrait.proto.SimpleExtensionDeclaration
import scala.collection.JavaConverters._
import scala.collection.mutable.ArrayBuffer
-class VeloxValidatorApi extends ValidatorApi {
+class VeloxValidatorApi extends ValidatorApi with Logging {
import VeloxValidatorApi._
- /** For velox backend, key validation is on native side. */
- override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = {
- expr match {
- // Both format_string and printf resolve to Spark's FormatString
expression.
- case formatString: FormatString =>
- supportsFormatString(formatString)
- case _ => true
- }
+ /**
+ * Enforce integral ROUND eligibility on the JVM to preserve Spark's
captured ANSI semantics,
+ * independently of optional native plan validation.
+ */
+ override def doExprValidate(substraitExprName: String, expr: Expression):
Boolean = expr match {
+ // Both format_string and printf resolve to Spark's FormatString
expression.
+ case formatString: FormatString =>
+ supportsFormatString(formatString)
+ case round: Round =>
+ round.child.dataType match {
+ case ByteType | ShortType | IntegerType | LongType =>
+ // Negative scales can overflow, and Spark captures ANSI mode in the
expression.
+ val supported = round.scale match {
+ case Literal(null, IntegerType) => true
+ case Literal(scale: Int, IntegerType) =>
+ scale >= 0 && scale <= MaxQualifiedIntegralRoundScale
+ case _ => false
+ }
+ if (!supported) {
+ logDebug(
+ "Integral ROUND requires a folded INTEGER scale in " +
+ s"[0, $MaxQualifiedIntegralRoundScale] or NULL; retaining
Spark.")
+ }
+ supported
+ case _ => true
+ }
+ case _ => true
}
Review Comment:
The PR description states that disabling native validation is a debugging
mode that “bypasses this compatibility fallback”, but the JVM-side
`doExprValidate` enforces Spark fallback for integral ROUND regardless of
`spark.gluten.sql.nativeValidation.enabled` (and the docs update shown also
doesn’t mention any bypass). Please reconcile this by either (a) updating the
PR description/docs to reflect that ROUND fallback is always enforced on the
JVM, or (b) gating this ROUND-specific validation behind the intended config if
the bypass is desired.
##########
docs/backends/velox/velox-backend-scalar-function-support.md:
##########
@@ -297,7 +297,7 @@ The status applies to `spark.sql.ansi.enabled=false`. When
ANSI mode is enabled,
| randn | Randn | S | |
| random | Rand | S | |
| rint | Rint | S | |
-| round | Round | S | |
+| round | Round | PS | Integral scales in
[0, 400] and NULL use upstream Velox. NULL scales skip child evaluation.
Negative scales, scales outside [0, 400], and scales not folded to INTEGER
literals fall back to Spark to preserve captured ANSI behavior. |
Review Comment:
This table cell is very long and hard to scan in rendered Markdown. Consider
shortening it (e.g., “Integral: native for scale ∈ [0,400] or NULL; otherwise
Spark fallback”) and, if needed, linking to a short dedicated subsection that
explains the full rationale (captured ANSI behavior, non-literal scales, NULL
short-circuiting).
##########
cpp/velox/tests/SparkFunctionTest.cc:
##########
@@ -125,6 +126,114 @@ TEST_F(SparkFunctionTest, roundWithDecimal) {
runRoundWithDecimalTest<int8_t>(testRoundWithDecIntegralData<int8_t>());
}
+TEST_F(SparkFunctionTest, integralRoundUsesSessionAnsi) {
+ const auto input = makeRowVector({makeNullableFlatVector<int8_t>({127, -128,
25, -25, std::nullopt})});
+ for (const auto ansi : {false, true}) {
+ queryCtx_->testingOverrideConfigUnsafe({{sparkAnsiEnabledConfigKey(), ansi
? "true" : "false"}});
+ const auto expression = "round(c0, cast(-1 as integer))";
+ if (ansi) {
+ VELOX_ASSERT_USER_THROW(evaluate(expression, input), "Arithmetic
overflow: round(");
Review Comment:
This assertion is brittle because it depends on an exact error-message
substring (`"Arithmetic overflow: round("`), which can change across Velox
versions or formatting changes. Consider asserting on a less specific substring
(e.g., `"Arithmetic overflow"`) or using an assertion helper that checks error
type/code (if available in this test harness) to reduce false failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]