vb-dbrks commented on PR #59408: URL: https://github.com/apache/spark/pull/59408#issuecomment-6131540574
I took another look at the compatibility implications of the bug I reported and updated [SPARK-60202](https://issues.apache.org/jira/browse/SPARK-60202) and the PR description to make two points explicit: - The current candidate changes ordinary Float results too: `ceil(CAST('0.1' AS FLOAT), 1)` goes from `0.1` to `0.2`, and `floor(CAST('0.9' AS FLOAT), 1)` from `0.9` to `0.8`. This follows widening Float before decimal conversion, but it is a compatibility choice worth reviewing beyond the original Double examples. - `ceil(CAST('1e-20' AS DOUBLE), -38)` exposes a result overflow that the earlier cast hid by producing zero. The candidate throws with ANSI both enabled and disabled through the existing decimal rounding path. [Spark's general ANSI documentation](https://spark.apache.org/docs/latest/sql-ref-ansi-compliance.html#arithmetic-operations) describes null for decimal overflow with ANSI disabled, so my earlier claim that all overflow behavior was unchanged was too broad. I also checked the SQL-standard and other-engine context, which is now in the JIRA. That does not establish one required Float conversion policy for Spark's two-argument, decimal-returning form. Would you prefer to include these compatibility changes in this fix, or narrow its scope while we agree on the Float and ANSI-disabled overflow behavior? I haven't pushed any code changes from this follow-up review. The additional regression tests, independent schema assertions and migration note remain to be added once the intended behavior is settled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
