Spenserrrr opened a new pull request, #58266: URL: https://github.com/apache/spark/pull/58266
### What changes were proposed in this pull request? Add a `ValueError` branch to `_scalar_str` in `python/pyspark/testing/goldenutils.py` so a `time64[ns]` scalar holding INT64_MIN renders as the marker `NaT collision`, beside the existing `temporal overflow` and raw-bytes fallbacks, and regenerate the four affected `from_pandas` type-scalar golden cells (`int64:standard` and `Int64:standard` x `time64[ns]`, safe and unsafe). Rendering a `time64[ns]` scalar goes through pandas, because Python's `datetime.time` has only microsecond resolution. pandas reserves INT64_MIN as its NaT sentinel, so it refuses that one value with `ValueError: NaTType does not support time`. The branch is gated on both the type and the value, so any other `ValueError` still propagates. ### Why are the changes needed? `PyArrowArrayFromPandasTypeScalarTests` currently fails on master. The four cells above read `ERR@ValueError`, which by convention means the conversion raised — but the conversion succeeds and only the rendering failed. The broad `try/except` in the cell helper recorded the rendering failure as if it were a conversion error, so narrowing that `try/except` in #58091 turned the mislabelling into a test error. `int64:standard` is `[0, 1, -1, iinfo.max, iinfo.min]`, and `iinfo.min` is exactly `pd.NaT.value`. The source row is kept as is: INT64_MIN being reinterpreted as a pandas sentinel is the kind of upstream behaviour these goldens exist to pin. ### Does this PR introduce _any_ user-facing change? No. Test-only. ### How was this patch tested? - `python/pyspark/tests/upstream/pyarrow/` passes (63 passed, 1 skipped). - Swept pyarrow 18-25 x pandas 2/3, 16/16 pass with no `overrides` entries, so the pandas delegation and the sentinel value are stable across the supported range. - The regenerated diff is exactly the four cells; row names and column headers are byte-identical. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Claude Opus 5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
