[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40520: [SPARK-42896][SQL][PYSPARK] Make `mapInPandas` / `mapInArrow` support barrier mode execution

HyukjinKwon commented on code in PR #40520: URL: https://github.com/apache/spark/pull/40520#discussion_r1145632491 ## python/pyspark/sql/pandas/map_ops.py: ## @@ -60,6 +62,7 @@ def mapInPandas( schema : :class:`pyspark.sql.types.DataType` or str the return

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40520: [SPARK-42896][SQL][PYSPARK] Make `mapInPandas` / `mapInArrow` support barrier mode execution

HyukjinKwon commented on code in PR #40520: URL: https://github.com/apache/spark/pull/40520#discussion_r1145632294 ## python/pyspark/sql/pandas/map_ops.py: ## @@ -32,7 +32,9 @@ class PandasMapOpsMixin: """ def mapInPandas( -self, func:

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40520: [SPARK-42896][SQL][PYSPARK] Make `mapInPandas` / `mapInArrow` support barrier mode execution

HyukjinKwon commented on code in PR #40520: URL: https://github.com/apache/spark/pull/40520#discussion_r1144849964 ## sql/core/src/main/scala/org/apache/spark/sql/execution/python/MapInPandasExec.scala: ## @@ -28,7 +28,8 @@ import org.apache.spark.sql.execution.SparkPlan case

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40520: [SPARK-42896][SQL][PYSPARK] Make `mapInPandas` / `mapInArrow` support barrier mode execution

HyukjinKwon commented on code in PR #40520: URL: https://github.com/apache/spark/pull/40520#discussion_r1144848610 ## sql/core/src/main/scala/org/apache/spark/sql/execution/python/MapInPandasExec.scala: ## @@ -28,7 +28,8 @@ import org.apache.spark.sql.execution.SparkPlan case

[GitHub] [spark] HyukjinKwon commented on a diff in pull request #40520: [SPARK-42896][SQL][PYSPARK] Make `mapInPandas` / `mapInArrow` support barrier mode execution

HyukjinKwon commented on code in PR #40520: URL: https://github.com/apache/spark/pull/40520#discussion_r1144800754 ## sql/core/src/main/scala/org/apache/spark/sql/execution/python/MapInPandasExec.scala: ## @@ -28,7 +28,8 @@ import org.apache.spark.sql.execution.SparkPlan case