viirya commented on PR #59334: URL: https://github.com/apache/spark/pull/59334#issuecomment-6100496414
Thanks for the thorough review, @dongjoon-hyun. - Large negative exponents (`'1e-2147483647'`, and the slow `'1e-10000000'`) and the legacy negative-scale conf: these fail in `changePrecision` (`setScale` with a huge scale delta), not in the fast-fail check this PR fixes. They are handled in #59335 (SPARK-60119), together with cast-level tests for both cases, so I kept this PR scoped to the fast-fail. - Zero: agreed, `'0e39'` returning NULL is the same false positive. The fast-fail now skips zero (`bigDecimal.signum != 0`), so `'0e2147483647'` keeps returning `0.00` and `'0e39'` is fixed. I updated the tests and the PR description. - `Decimal.set` L152 and the other `BigDecimal` entry points: filed SPARK-60132. For CSV/JSON the exception currently becomes a bad record rather than a task failure, but it should still be handled consistently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
