Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22807#discussion_r227784077
--- Diff: python/pyspark/sql/tests.py ---
@@ -4961,6 +4961,31 @@ def foofoo(x, y):
).collect
)
+def
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22807#discussion_r227611310
--- Diff: python/pyspark/serializers.py ---
@@ -248,7 +248,14 @@ def create_array(s, t):
# TODO: see ARROW-2432. Remove when the minimum
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22807#discussion_r227611338
--- Diff: python/pyspark/sql/tests.py ---
@@ -4961,6 +4961,31 @@ def foofoo(x, y):
).collect
)
+def
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/22807#discussion_r227571481
--- Diff: python/pyspark/serializers.py ---
@@ -248,7 +248,14 @@ def create_array(s, t):
# TODO: see ARROW-2432. Remove when the
Github user BryanCutler commented on a diff in the pull request:
https://github.com/apache/spark/pull/22807#discussion_r227572701
--- Diff: python/pyspark/sql/tests.py ---
@@ -4961,6 +4961,31 @@ def foofoo(x, y):
).collect
)
+def
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/22807
[WIP][SPARK-25811][PySpark] Raise a proper error when unsafe cast is
detected by PyArrow
## What changes were proposed in this pull request?
Since 0.11.0, PyArrow supports to raise an
