This is an automated email from the ASF dual-hosted git repository. gurwls223 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/spark.git.
from 0a4f985c [SPARK-23098][SQL] Migrate Kafka Batch source to v2. add 02f4763 [SPARK-28198][PYTHON] Add mapPartitionsInPandas to allow an iterator of DataFrames No new revisions were added by this update. Summary of changes: .../org/apache/spark/api/python/PythonRunner.scala | 2 + python/pyspark/rdd.py | 1 + python/pyspark/sql/dataframe.py | 48 +++++++- python/pyspark/sql/tests/test_pandas_udf_iter.py | 135 +++++++++++++++++++++ python/pyspark/worker.py | 62 ++++++---- .../plans/logical/pythonLogicalOperators.scala | 12 ++ .../main/scala/org/apache/spark/sql/Dataset.scala | 21 +++- .../spark/sql/execution/SparkStrategies.scala | 2 + .../python/MapPartitionsInPandasExec.scala | 95 +++++++++++++++ 9 files changed, 353 insertions(+), 25 deletions(-) create mode 100644 python/pyspark/sql/tests/test_pandas_udf_iter.py create mode 100644 sql/core/src/main/scala/org/apache/spark/sql/execution/python/MapPartitionsInPandasExec.scala --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org