luis4a0 opened a new pull request, #13047:
URL: https://github.com/apache/gluten/pull/13047
## What changes are proposed in this pull request?
Wire Spark BROUND to the corresponding Velox functions, including routing
decimal inputs to `decimal_bround`.
Integral calls carry the Spark expression's captured ANSI flag as a native
constant. This preserves overflow behavior even when a DataFrame is analyzed
or physically planned before the session mode changes.
Add planning-time compatibility checks:
- Scales outside the native **[-400, 400]** interval fall back to Spark.
Checking
before native execution is necessary because primitive initialization
errors
can be deferred.
- Nonzero-scale FLOAT/DOUBLE BROUND falls back on JVMs older than Java 21 to
preserve their decimal-conversion behavior. Scale-zero floating rounding
and
integral/decimal calls remain eligible for native execution.
- NULL INTEGER scales are accepted. Scales not folded to INTEGER literals
retain safe fallback.
Existing ROUND behavior is unchanged.
**Dependency:** this draft requires the companion upstream Velox BROUND
change
to be included in Gluten's pinned Velox dependency before merge.
https://github.com/facebookincubator/velox/pull/19072
The integration was built and exercised against public IBM Velox
`dft-2026_09_14` with that same production patch applied:
https://github.com/IBM/velox/tree/dft-2026_09_14
The stock tag does not yet contain BROUND. This PR does not claim otherwise
or
replace the dependency pin with an unrelated upstream build.
## How was this patch tested?
- Fresh native backend and public-dependency Spark 4.1 / Scala 2.13 builds.
- Five focused native ROUND/BROUND execution and mapping tests passed, in
addition to the earlier parser regressions. The new tests first failed
against the baseline: missing BROUND registration and incorrect decimal
function mapping.
- Eight validator unit tests and two mode-serialization unit tests passed.
- Nine Spark 4.1 integration tests passed on both Java 17 and Java 21. They
assert the actual native BROUND projection or the intentional fallback,
compare schemas and floating-point bits, and exercise integral overflow,
decimal precision/overflow, folded and NULL scales (including native FLOAT
and DOUBLE NULL-scale calls on both JVMs and execution modes), unsupported
scales,
and mode changes after analysis and after cached native physical planning.
A remaining Spark BROUND in a partially native projection fails the check.
- The companion native implementation matched registered public Spark 4.1.1
interpreted/generated expressions on **5,535,698 distinct typed inputs**
(**11,071,396 LEGACY/ANSI mode-cases per flat and
dictionary/partial-selection
shape**) with zero mismatches. Another 586,332 mode-cases per shape covered
every supported scale.
- The expression-mode overload also passed 5,452,400 integral evaluations
with
the native query mode deliberately set opposite to the captured mode.
The multi-million floating campaign uses JDK 21. A separate actual-JVM
comparison established the older-JDK differences that motivate the fallback;
461,532 native-eligible dense-scale cases matched between JDK 17 and JDK 21.
The evidence does not claim universal cross-JDK floating equivalence, exact
error-message equivalence, exhaustive correctness, or performance gains.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI 1.0.86-2; OpenAI models.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
