Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18664
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147207020
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147126818
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147126889
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -1880,11 +1880,13 @@ def toPandas(self):
import pandas as pd
if
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147127081
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147024180
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147023750
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147021983
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147021068
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147020254
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,38 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r147019262
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146926543
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146923941
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146922937
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146747982
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146651671
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146641097
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146636005
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,33 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146567335
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146472528
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,33 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146472494
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146472109
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146469109
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146419125
--- Diff: python/pyspark/serializers.py ---
@@ -224,7 +225,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146347107
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146096087
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146066999
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146062157
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146058020
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r146044463
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145902248
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145901149
--- Diff: python/pyspark/serializers.py ---
@@ -223,12 +224,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145786734
--- Diff: python/pyspark/sql/tests.py ---
@@ -3383,6 +3403,42 @@ def test_vectorized_udf_varargs(self):
res = df.select(f(col('id')))
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145783400
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145737159
--- Diff: python/pyspark/sql/tests.py ---
@@ -3383,6 +3403,42 @@ def test_vectorized_udf_varargs(self):
res = df.select(f(col('id')))
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145729785
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -42,6 +43,13 @@ object ArrowUtils {
case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145725470
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -42,6 +43,13 @@ object ArrowUtils {
case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145724885
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -42,6 +43,13 @@ object ArrowUtils {
case
Github user jreback commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145654005
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user jreback commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145653001
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145597733
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -42,6 +43,13 @@ object ArrowUtils {
case
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145597432
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowUtils.scala
---
@@ -42,6 +43,13 @@ object ArrowUtils {
case
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145486068
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145485217
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145414587
--- Diff: python/pyspark/serializers.py ---
@@ -223,12 +224,13 @@ def _create_batch(series):
# If a nullable integer series has been promoted to
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145415365
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145415564
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145415575
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user wesm commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145415011
--- Diff: python/pyspark/sql/types.py ---
@@ -1619,11 +1619,39 @@ def to_arrow_type(dt):
arrow_type = pa.decimal(dt.precision, dt.scale)
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/18664#discussion_r145327316
--- Diff: python/pyspark/serializers.py ---
@@ -259,11 +261,13 @@ def load_stream(self, stream):
"""
Deserialize ArrowRecordBatches
50 matches
Mail list logo
