luis4a0 commented on PR #13103: URL: https://github.com/apache/gluten/pull/13103#issuecomment-6653328691
@rui-mo, continuing the offload-boundary discussion from: https://github.com/facebookincubator/velox/pull/19141#issuecomment-6637881984 I agree this belongs in the Gluten companion and should not block the native Velox PR. Velox can now execute every constant INTEGER scale. Gluten still needs to preserve Spark result metadata and execution behavior. One known incompatibility is at scales `INT32_MIN` and `INT32_MIN + 1`: Velox requires `DECIMAL(38,0)`, while Spark 4.1's 32-bit metadata arithmetic resolves `DECIMAL(min(p - s + 1, 38), 0)` for `DECIMAL(p,s)`. I propose qualifying the actual registered Spark 4.1 and Velox functions across representative extreme scales, then either: 1. remove the bound with explicit exclusions for incompatible endpoints; or 2. retain a narrower documented compatibility interval if JVM execution differs more broadly. Until that comparison is complete, the current `[-400,400]` guard remains a conservative compatibility policy, not a Velox capability limit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
