[
https://issues.apache.org/jira/browse/SPARK-54481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-54481.
----------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Issue resolved by pull request 53235
[https://github.com/apache/spark/pull/53235]
> Reenable test_apply_in_pandas_returning_incompatible_type
> ---------------------------------------------------------
>
> Key: SPARK-54481
> URL: https://issues.apache.org/jira/browse/SPARK-54481
> Project: Spark
> Issue Type: Sub-task
> Components: Tests
> Affects Versions: 4.1.0, 4.0.1, 4.2.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> {code}
> ======================================================================
> FAIL [0.000s]: test_apply_in_pandas_returning_incompatible_type
> (pyspark.sql.tests.connect.pandas.test_parity_pandas_cogrouped_map.CogroupedApplyInPandasTests.test_apply_in_pandas_returning_incompatible_type)
> [without key] (convert='string to double', convertToArrowArraySafely=True)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 574, in _test_merge_error
> self.__test_merge_error(
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 606, in __test_merge_error
> self.__test_merge(left, right, by, fn, output_schema)
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 557, in __test_merge
> assert_frame_equal(expected, result)
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 1218, in assert_frame_equal
> raise_assert_detail(
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 614, in raise_assert_detail
> raise AssertionError(msg)
> AssertionError: DataFrame are different
> DataFrame shape mismatch
> [left]: (100, 4)
> [right]: (10, 2)
> ======================================================================
> FAIL [0.000s]: test_apply_in_pandas_returning_incompatible_type
> (pyspark.sql.tests.connect.pandas.test_parity_pandas_cogrouped_map.CogroupedApplyInPandasTests.test_apply_in_pandas_returning_incompatible_type)
> [with key] (convert='string to double', convertToArrowArraySafely=True)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 584, in _test_merge_error
> self.__test_merge_error(
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 606, in __test_merge_error
> self.__test_merge(left, right, by, fn, output_schema)
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 557, in __test_merge
> assert_frame_equal(expected, result)
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 1218, in assert_frame_equal
> raise_assert_detail(
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 614, in raise_assert_detail
> raise AssertionError(msg)
> AssertionError: DataFrame are different
> DataFrame shape mismatch
> [left]: (100, 4)
> [right]: (10, 2)
> ======================================================================
> FAIL [0.000s]: test_apply_in_pandas_returning_incompatible_type
> (pyspark.sql.tests.connect.pandas.test_parity_pandas_cogrouped_map.CogroupedApplyInPandasTests.test_apply_in_pandas_returning_incompatible_type)
> [without key] (convert='string to double', convertToArrowArraySafely=False)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 574, in _test_merge_error
> self.__test_merge_error(
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 606, in __test_merge_error
> self.__test_merge(left, right, by, fn, output_schema)
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 557, in __test_merge
> assert_frame_equal(expected, result)
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 1218, in assert_frame_equal
> raise_assert_detail(
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 614, in raise_assert_detail
> raise AssertionError(msg)
> AssertionError: DataFrame are different
> DataFrame shape mismatch
> [left]: (100, 4)
> [right]: (10, 2)
> ======================================================================
> FAIL [3.160s]: test_apply_in_pandas_returning_incompatible_type
> (pyspark.sql.tests.connect.pandas.test_parity_pandas_cogrouped_map.CogroupedApplyInPandasTests.test_apply_in_pandas_returning_incompatible_type)
> [with key] (convert='string to double', convertToArrowArraySafely=False)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 584, in _test_merge_error
> self.__test_merge_error(
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 606, in __test_merge_error
> self.__test_merge(left, right, by, fn, output_schema)
> File
> "/home/runner/work/spark/spark-4.0/python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py",
> line 557, in __test_merge
> assert_frame_equal(expected, result)
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 1218, in assert_frame_equal
> raise_assert_detail(
> File
> "/opt/hostedtoolcache/Python/3.11.14/x64/lib/python3.11/site-packages/pandas/_testing/asserters.py",
> line 614, in raise_assert_detail
> raise AssertionError(msg)
> AssertionError: DataFrame are different
> DataFrame shape mismatch
> [left]: (100, 4)
> [right]: (10, 2)
> ----------------------------------------------------------------------
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
