HyukjinKwon opened a new pull request, #316: URL: https://github.com/apache/spark-connect-rust/pull/316
### What changes were proposed in this pull request? [SPARK-61121](https://issues.apache.org/jira/browse/SPARK-61121): A float bound of `rowsBetween` and `rangeBetween` past the threshold of its side is unbounded, as PySpark's. PySpark compares the bounds with `Window._PRECEDING_THRESHOLD` and `_FOLLOWING_THRESHOLD` before `WindowFrame` asserts an int: - A start at or below the first is unbounded preceding, also for a float. - An end at or above the second is unbounded following, also for a float. So `w.rowsBetween(float("-inf"), float("inf"))` is a frame of unbounded bounds. The client now does the same, and still raises `AssertionError` for another float, such as `1.5`, a start of `inf` or an end of `-inf`. ### Why are the changes needed? The client raised `AssertionError` for any float bound, also an infinite one. Apache Spark's `test_window_functions` and `test_window_functions_without_partitionBy` use infinite bounds, and failed on this client. ### Does this PR introduce _any_ user-facing change? Yes. Infinite and other out-of-range float bounds are unbounded, as with PySpark 4.3.0. ### How was this patch tested? - The offline test of window frames checks: - the frames of infinite and out-of-range float bounds, which are those of `unboundedPreceding` and `unboundedFollowing` - the `AssertionError` of other floats The cases give the same results with official PySpark 4.3.0. - Apache Spark's `test_parity_functions.py`, run on this client against a local server, now passes `test_window_functions` and `test_window_functions_without_partitionBy`. - The lib tests of both crates (590 passed), the build of all tests, also with `wasm-udf`, the offline Python suites (2165 passed, 8 skipped), and the live drop-in and Arrow suites (173 passed) and live cases (10 passed) against a local Spark 4.3.0 server. Clippy reports no diagnostics in the changed lines. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
