cloud-fan commented on code in PR #58408:
URL: https://github.com/apache/spark/pull/58408#discussion_r3996976265
##########
docs/sql-migration-guide.md:
##########
@@ -25,6 +25,7 @@ license: |
## Upgrading from Spark SQL 4.3 to 4.4
- Since Spark 4.4, the optimizer rule `SimplifyCaseConversionExpressions` no
longer simplifies mixed case conversions `LOWER(UPPER(str))` to `LOWER(str)` or
`UPPER(LOWER(str))` to `UPPER(str)`. This preserves standard Unicode case
mapping semantics for non-ASCII characters such as Latin small letter dotless i
(`ı` U+0131) and micro sign (`µ` U+00B5).
+- Since Spark 4.4, `format_string` and `printf` pass a decimal argument to
`java.util.Formatter` as a `java.math.BigDecimal`, so the floating-point
conversions `%f`, `%e`, and `%g` format decimals instead of failing with
`IllegalFormatConversionException`. `%a` still rejects a decimal, because
`Formatter` supports it only for `float` and `double`. As a side effect, `%h`
on a decimal prints a different hash than earlier releases, because it now
hashes the `java.math.BigDecimal` (unscaled value and scale) rather than
Spark's internal `Decimal` (which hashes consistently with `Double`). `%s`
output and `NULL` handling are unchanged.
Review Comment:
Confirmed: the migration note now says the hash can change and names a case
where it does not.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"inline:3926331028","thread_id":"inline:3926331028","verdict_sha256":"8962b2f244b4a8a02254ee3e94fbb0fc1f898f281cfa0aba7c9b69ab4915925b"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
