Spenserrrr opened a new pull request, #58257: URL: https://github.com/apache/spark/pull/58257
### What changes were proposed in this pull request? Skip the NumPy-comparison tests for `reciprocal` and `fmax`/`fmin` unless running on Linux x86-64 with NumPy >= 2.3.0, and exclude `reciprocal` from the random-integer sweeps (`test_np_spark_compat_series` / `_frame`) elsewhere. Test-only change; `numpy_compat.py` is untouched. This is an alternative to #58019, which instead changed the implementation to fall back to `pandas_udf` off that platform. ### Why are the changes needed? `np.reciprocal(int 0)` and the `fmax`/`fmin` signed-zero tie are unspecified by C/IEEE, so NumPy's own answer varies by CPU architecture (x86-64 casts the intermediate `+inf` to `INT64_MIN`, aarch64 saturates to `INT64_MAX`) and by NumPy version (the scalar tie returns the first operand from 2.3.0 on, the second before). pandas-on-Spark deliberately returns one fixed value on every platform, which is the desired cross-platform-consistent behavior. The locally installed NumPy is therefore only a valid reference on the environment that value was verified against, so the scheduled ARM and macOS `pyspark-pandas` jobs currently fail on `test_np_fmax_fmin`, `test_np_reciprocal_integer`, and intermittently `test_np_spark_compat_series` (its random data sometimes draws a 0). ### Does this PR introduce _any_ user-facing change? No. Tests only. ### How was this patch tested? Existing tests. On Linux x86-64 with NumPy 2.4.1 both tests still run and pass; with `platform.machine`, `platform.system`, and `np.__version__` patched to macOS arm64, Linux aarch64, and NumPy 1.23.2, they are skipped as intended. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (2.1.241) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
