Xinrong Meng created SPARK-36274: ------------------------------------ Summary: Fix equality comparison of unordered Categoricals Key: SPARK-36274 URL: https://issues.apache.org/jira/browse/SPARK-36274 Project: Spark Issue Type: Sub-task Components: PySpark Affects Versions: 3.2.0 Reporter: Xinrong Meng
We cannot rely on codes when compare equality of unordered Categoricals. An example looks like {code:java} >>> (ps.Series(pd.Categorical(list('abca'))) == >>> ps.Series(pd.Categorical(list('bcaa'), >>> categories=list('bca')))).sort_index() 0 True 1 True 2 True 3 False dtype: bool {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org