[GitHub] [spark] itholic commented on a diff in pull request #42793: [SPARK-45065][PYTHON][PS] Support Pandas 2.1.0

2023-09-16 Thread via GitHub
itholic commented on code in PR #42793: URL: https://github.com/apache/spark/pull/42793#discussion_r1328008105 ## python/docs/source/migration_guide/pyspark_upgrade.rst: ## @@ -42,6 +42,8 @@ Upgrading from PySpark 3.5 to 4.0 * In Spark 4.0, ``squeeze`` parameter from

[GitHub] [spark] itholic commented on a diff in pull request #42793: [SPARK-45065][PYTHON][PS] Support Pandas 2.1.0

2023-09-15 Thread via GitHub
itholic commented on code in PR #42793: URL: https://github.com/apache/spark/pull/42793#discussion_r1327908926 ## python/pyspark/pandas/frame.py: ## @@ -1321,11 +1323,76 @@ def applymap(self, func: Callable[[Any], Any]) -> "DataFrame": 0 1.00 4.494400