Yicong-Huang opened a new pull request, #58259: URL: https://github.com/apache/spark/pull/58259
### What changes were proposed in this pull request? This is a follow-up to [SPARK-46161](https://issues.apache.org/jira/browse/SPARK-46161), which added `axis=1` support to pandas-on-Spark `DataFrame.diff`. The new `axis=1` branch skipped the `periods` type validation that the `axis=0` path already enforces (via `Series._diff`). This PR adds the same guard at the top of the `axis=1` branch so a non-integer `periods` raises the established `TypeError` instead of a raw Python list-index error. ### Why are the changes needed? The `axis=0` path validates `periods` in `Series._diff` and raises `TypeError("periods should be an int; however, got [...]")`. The `axis=1` branch computed `prev_idx = i - periods` and then indexed `column_labels[prev_idx]`, so a non-integer `periods` leaked an unhelpful Python error rather than the documented one. For example, before this change: ```python >>> import pyspark.pandas as ps >>> psdf = ps.DataFrame({"a": [1, 2, 3], "b": [1, 2, 3], "c": [1, 2, 3]}) >>> psdf.diff(1.5, axis=1) TypeError: list indices must be integers or slices, not float ``` After this change it matches the `axis=0` behavior: ```python >>> psdf.diff(1.5, axis=1) TypeError: periods should be an int; however, got [float] ``` ### Does this PR introduce _any_ user-facing change? Yes, but only within the unreleased `axis=1` support added by SPARK-46161. Passing a non-integer `periods` with `axis=1` now raises `TypeError: periods should be an int; however, got [...]` instead of `TypeError: list indices must be integers or slices, not float`. There is no change compared to released Spark versions, and valid calls are unaffected. ### How was this patch tested? Added a negative assertion to `test_diff` in `python/pyspark/pandas/tests/computation/test_compute.py` covering `psdf.diff(1.5, axis=1)`, mirroring the existing `axis=0` check. ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
