[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606090340 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -20,6 +20,9 @@ """ from pyspark.serializers import Serializer, read_int, write_int,

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606088240 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala ## @@ -89,9 +89,57 @@ case class

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606075743 ## File path: python/pyspark/sql/tests/test_arrow.py ## @@ -196,6 +197,33 @@ def test_pandas_round_trip(self): pdf_arrow = df.toPandas()

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606039240 ## File path: python/pyspark/sql/tests/test_arrow.py ## @@ -196,6 +197,33 @@ def test_pandas_round_trip(self): pdf_arrow = df.toPandas()

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606036073 ## File path: python/pyspark/sql/tests/test_arrow.py ## @@ -196,6 +197,33 @@ def test_pandas_round_trip(self): pdf_arrow = df.toPandas()

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606035670 ## File path: python/pyspark/sql/types.py ## @@ -764,6 +764,21 @@ def __eq__(self, other): return type(self) == type(other) +def

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606035644 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala ## @@ -89,9 +89,57 @@ case class

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606026938 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -452,24 +457,27 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone):

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606026544 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -20,9 +20,10 @@ from pyspark.rdd import _load_from_socket from

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r606025117 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -452,24 +457,27 @@ def _create_from_pandas_with_arrow(self, pdf, schema, timezone):

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r60566 ## File path: python/pyspark/sql/pandas/types.py ## @@ -346,3 +348,29 @@ def _convert_dict_to_map_items(s): :return: pandas.Series of lists of (key,

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r60566 ## File path: python/pyspark/sql/pandas/types.py ## @@ -346,3 +348,29 @@ def _convert_dict_to_map_items(s): :return: pandas.Series of lists of (key,

[GitHub] [spark] sadhen commented on a change in pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

sadhen commented on a change in pull request #32026: URL: https://github.com/apache/spark/pull/32026#discussion_r60566 ## File path: python/pyspark/sql/pandas/types.py ## @@ -346,3 +348,29 @@ def _convert_dict_to_map_items(s): :return: pandas.Series of lists of (key,