adiranjan25 opened a new pull request, #60016: URL: https://github.com/apache/spark/pull/60016
## What changes were proposed in this pull request? This PR fixes pandas-on-Spark's handling of `np.floor` for floating-point Series. The previous implementation delegates directly to Spark SQL's `floor` function, which produces an integral result. This can cause arithmetic overflow for infinity and large floating-point values and does not preserve negative zero. The change introduces a floating-point-specific expression that: - Preserves positive and negative infinity. - Preserves NaN and signed zero. - Avoids integral overflow for large floating-point values that are already integral at double precision. - Retains the existing behavior for integer and decimal inputs, avoiding unnecessary loss of decimal precision. ## How was this patch tested? Added regression tests covering: - Positive and negative infinity - NaN - Large positive and negative floating-point values - Positive and negative fractional values - Positive and negative zero - Float32 and float64 inputs - Integer and decimal compatibility Ran the pandas-on-Spark NumPy compatibility test class locally: `Ran 34 tests in 24.302s` `OK (skipped=3)` Also verified `git diff --check`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
