Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19349
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141257653
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141257047
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141256430
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141256118
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141250276
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/PythonUDFRunner.scala
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141250251
--- Diff: python/pyspark/serializers.py ---
@@ -251,6 +256,36 @@ def __repr__(self):
return "ArrowPandasSerializer"
+class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141250303
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -0,0 +1,429 @@
+/*
+ * Licensed to the Apache Software
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141250227
--- Diff: python/pyspark/serializers.py ---
@@ -211,33 +212,37 @@ def __repr__(self):
return "ArrowSerializer"
+def
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141250242
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141246604
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141244651
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala ---
@@ -0,0 +1,429 @@
+/*
+ * Licensed to the Apache Software
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141243843
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141242888
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141242553
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/PythonUDFRunner.scala
---
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141242240
--- Diff: python/pyspark/serializers.py ---
@@ -251,6 +256,36 @@ def __repr__(self):
return "ArrowPandasSerializer"
+class
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141240246
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141237789
--- Diff: python/pyspark/serializers.py ---
@@ -211,33 +212,37 @@ def __repr__(self):
return "ArrowSerializer"
+def
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141142944
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ *
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141105065
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141104830
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowStreamPythonUDFRunner.scala
---
@@ -0,0 +1,197 @@
+/*
+ * Licensed
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141103372
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -925,6 +925,13 @@ object SQLConf {
.intConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19349#discussion_r141101690
--- Diff: python/pyspark/serializers.py ---
@@ -251,6 +256,36 @@ def __repr__(self):
return "ArrowPandasSerializer"
+class
GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/19349
[SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format for vectorized UDF.
## What changes were proposed in this pull request?
Currently we use Arrow File format to communicate with Python
24 matches
Mail list logo
