Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r152201452
--- Diff: python/pyspark/sql/types.py ---
@@ -1678,37 +1678,105 @@ def from_arrow_schema(arrow_schema):
for field in arrow_schema])
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r152199700
--- Diff: python/pyspark/sql/types.py ---
@@ -1678,37 +1678,105 @@ def from_arrow_schema(arrow_schema):
for field in arrow_schema])
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r152174929
--- Diff: python/pyspark/worker.py ---
@@ -150,7 +150,8 @@ def read_udfs(pickleSer, infile, eval_type):
if eval_type ==
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r152174935
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -58,6 +59,11 @@ class ArrowPythonRunner(
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r152174813
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -1913,7 +1920,16 @@ def toPandas(self):
for f, t in dtype.items():
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r151999012
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowPythonRunner.scala
---
@@ -58,6 +59,11 @@ class ArrowPythonRunner(
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r151998797
--- Diff: python/pyspark/worker.py ---
@@ -150,7 +150,8 @@ def read_udfs(pickleSer, infile, eval_type):
if eval_type ==
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r151998296
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -1913,7 +1920,16 @@ def toPandas(self):
for f, t in dtype.items():
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149661483
--- Diff: python/pyspark/sql/session.py ---
@@ -557,7 +577,13 @@ def createDataFrame(self, data, schema=None,
samplingRatio=None, verifySchema=Tr
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149655172
--- Diff: python/pyspark/sql/session.py ---
@@ -557,7 +577,13 @@ def createDataFrame(self, data, schema=None,
samplingRatio=None, verifySchema=Tr
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149582142
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,37 +1629,82 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149055095
--- Diff: python/pyspark/serializers.py ---
@@ -274,12 +278,13 @@ def load_stream(self, stream):
"""
Deserialize ArrowRecordBatches
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149005432
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,112 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149005429
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,112 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149002542
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,112 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r149001569
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,112 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148601935
--- Diff: python/pyspark/serializers.py ---
@@ -274,12 +278,13 @@ def load_stream(self, stream):
"""
Deserialize
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148598592
--- Diff: python/pyspark/serializers.py ---
@@ -274,12 +278,13 @@ def load_stream(self, stream):
"""
Deserialize
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148576712
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -948,6 +948,14 @@ object SQLConf {
.intConf
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148576678
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,121 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148576656
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,121 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148541382
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -948,6 +948,14 @@ object SQLConf {
.intConf
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148540021
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,121 @@ def to_arrow_type(dt):
return arrow_type
-def
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19607#discussion_r148538507
--- Diff: python/pyspark/sql/types.py ---
@@ -1629,35 +1629,121 @@ def to_arrow_type(dt):
return arrow_type
-def
24 matches
Mail list logo
