This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 714fc8cd872d [SPARK-48316][PS][CONNECT][TESTS] Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition 714fc8cd872d is described below commit 714fc8cd872d6f583a6066e9ddb4a51caa51caf3 Author: Hyukjin Kwon <gurwls...@apache.org> AuthorDate: Fri May 17 12:09:49 2024 +0900 [SPARK-48316][PS][CONNECT][TESTS] Fix comments for SparkFrameMethodsParityTests.test_coalesce and test_repartition ### What changes were proposed in this pull request? This PR proposes to enable `SparkFrameMethodsParityTests.test_coalesce` and `SparkFrameMethodsParityTests.test_repartition` in Spark Connect by avoiding RDD usage in the test. ### Why are the changes needed? To make sure on the test coverage ### Does this PR introduce _any_ user-facing change? No, test-only. ### How was this patch tested? CI in this PR. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46629 from HyukjinKwon/SPARK-48316. Authored-by: Hyukjin Kwon <gurwls...@apache.org> Signed-off-by: Hyukjin Kwon <gurwls...@apache.org> --- python/pyspark/pandas/tests/connect/test_parity_frame_spark.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/python/pyspark/pandas/tests/connect/test_parity_frame_spark.py b/python/pyspark/pandas/tests/connect/test_parity_frame_spark.py index 24626a9164e8..c3672647b71b 100644 --- a/python/pyspark/pandas/tests/connect/test_parity_frame_spark.py +++ b/python/pyspark/pandas/tests/connect/test_parity_frame_spark.py @@ -28,7 +28,9 @@ class SparkFrameMethodsParityTests( def test_checkpoint(self): super().test_checkpoint() - @unittest.skip("Test depends on RDD which is not supported from Spark Connect.") + @unittest.skip( + "Test depends on RDD, and cannot use SQL expression due to Catalyst optimization" + ) def test_coalesce(self): super().test_coalesce() @@ -36,7 +38,9 @@ class SparkFrameMethodsParityTests( def test_local_checkpoint(self): super().test_local_checkpoint() - @unittest.skip("Test depends on RDD which is not supported from Spark Connect.") + @unittest.skip( + "Test depends on RDD, and cannot use SQL expression due to Catalyst optimization" + ) def test_repartition(self): super().test_repartition() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org