asl3 commented on code in PR #42332:
URL: https://github.com/apache/spark/pull/42332#discussion_r1284918439
##########
python/pyspark/testing/pandasutils.py:
##########
@@ -450,6 +477,15 @@ def assert_eq(self, left, right, check_exact=True,
almost=False):
import pandas as pd
from pandas.api.types import is_list_like
+ # for pandas-on-Spark, ignore row order by default because row
ordering not guaranteed
+ if isinstance(left, (ps.DataFrame, ps.Series, ps.Index)):
+ return assertPandasOnSparkEqual(
+ left, right, checkExact=check_exact, almost=almost,
checkRowOrder=check_row_order
Review Comment:
updated the above comment (allowing check_row_order=False for
pandas-on-Spark only)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
