Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r192344759
--- Diff: python/pyspark/tests.py ---
@@ -161,6 +161,37 @@ def gen_gs(N, step=1):
self.assertEqual(k, len(vs))
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21383
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191627958
--- Diff: python/pyspark/util.py ---
@@ -55,7 +55,9 @@ def _get_argspec(f):
"""
# `getargspec` is deprecated since python3.0
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191623502
--- Diff: python/pyspark/util.py ---
@@ -55,7 +55,9 @@ def _get_argspec(f):
"""
# `getargspec` is deprecated since python3.0
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191621339
--- Diff: python/pyspark/util.py ---
@@ -55,7 +55,9 @@ def _get_argspec(f):
"""
# `getargspec` is deprecated since python3.0
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191621391
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191619371
--- Diff: python/pyspark/util.py ---
@@ -55,7 +55,9 @@ def _get_argspec(f):
"""
# `getargspec` is deprecated since python3.0
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191619171
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191483040
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191481925
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191481196
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191473968
--- Diff: python/pyspark/sql/tests.py ---
@@ -900,6 +900,22 @@ def __call__(self, x):
self.assertEqual(f, f_.func)
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191472884
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191470695
--- Diff: python/pyspark/sql/tests.py ---
@@ -900,6 +900,22 @@ def __call__(self, x):
self.assertEqual(f, f_.func)
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191469832
--- Diff: python/pyspark/sql/tests.py ---
@@ -900,6 +900,22 @@ def __call__(self, x):
self.assertEqual(f, f_.func)
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191469562
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191441634
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191437141
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191427825
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191422043
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191418719
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191411253
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191409616
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191406867
--- Diff: python/pyspark/sql/udf.py ---
@@ -157,7 +157,17 @@ def _create_judf(self):
spark = SparkSession.builder.getOrCreate()
sc
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191113977
--- Diff: python/pyspark/util.py ---
@@ -89,6 +93,33 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r191069075
--- Diff: python/pyspark/util.py ---
@@ -89,6 +93,33 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
Github user icexelloss commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190761062
--- Diff: python/pyspark/sql/tests.py ---
@@ -900,6 +900,17 @@ def __call__(self, x):
self.assertEqual(f, f_.func)
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r19026
--- Diff: python/pyspark/sql/udf.py ---
@@ -92,7 +92,7 @@ def __init__(self, func,
raise TypeError(
"Invalid
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190666471
--- Diff: python/pyspark/sql/udf.py ---
@@ -92,7 +92,7 @@ def __init__(self, func,
raise TypeError(
"Invalid
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190664476
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190656662
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190607843
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190605953
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190603773
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190567010
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,31 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190598641
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,25 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190598227
--- Diff: python/pyspark/util.py ---
@@ -89,6 +89,23 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190453705
--- Diff: python/pyspark/rdd.py ---
@@ -791,9 +792,11 @@ def foreach(self, f):
>>> def f(x): print(x)
>>> sc.parallelize([1,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190450900
--- Diff: python/pyspark/sql/tests.py ---
@@ -900,6 +900,17 @@ def __call__(self, x):
self.assertEqual(f, f_.func)
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190450846
--- Diff: python/pyspark/tests.py ---
@@ -161,6 +161,37 @@ def gen_gs(N, step=1):
self.assertEqual(k, len(vs))
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190450814
--- Diff: python/pyspark/tests.py ---
@@ -161,6 +161,37 @@ def gen_gs(N, step=1):
self.assertEqual(k, len(vs))
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190450475
--- Diff: python/pyspark/util.py ---
@@ -89,6 +89,19 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190449941
--- Diff: python/pyspark/util.py ---
@@ -89,6 +89,19 @@ def majorMinorVersion(sparkVersion):
" version numbers.")
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190449695
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,31 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190449424
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,31 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190449208
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,31 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190448950
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1277,31 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190448642
--- Diff: python/pyspark/rdd.py ---
@@ -791,9 +792,11 @@ def foreach(self, f):
>>> def f(x): print(x)
>>> sc.parallelize([1,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190448318
--- Diff: python/pyspark/rdd.py ---
@@ -51,6 +51,7 @@
from pyspark.shuffle import Aggregator, ExternalMerger, \
get_used_memory,
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190448245
--- Diff: python/pyspark/rdd.py ---
@@ -1846,6 +1858,7 @@ def combineByKey(self, createCombiner, mergeValue,
mergeCombiners,
>>>
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190438434
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r190153031
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189967162
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189938003
--- Diff: python/pyspark/rdd.py ---
@@ -1846,6 +1858,7 @@ def combineByKey(self, createCombiner, mergeValue,
mergeCombiners,
>>>
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189903527
--- Diff: python/pyspark/tests.py ---
@@ -1246,6 +1286,37 @@ def test_pipe_unicode(self):
result = rdd.pipe('cat').collect()
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189903411
--- Diff: python/pyspark/tests.py ---
@@ -161,6 +161,46 @@ def gen_gs(N, step=1):
self.assertEqual(k, len(vs))
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189902156
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
--- End
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189901979
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
--- End
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189900650
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189890760
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189886781
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189886618
--- Diff: python/pyspark/rdd.py ---
@@ -173,6 +173,7 @@ def ignore_unicode_prefix(f):
return f
+
--- End diff --
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189872060
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189864634
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189863049
--- Diff: python/pyspark/rdd.py ---
@@ -173,6 +173,7 @@ def ignore_unicode_prefix(f):
return f
+
--- End diff --
I
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189824324
--- Diff: python/pyspark/rdd.py ---
@@ -173,6 +173,7 @@ def ignore_unicode_prefix(f):
return f
+
--- End diff --
Github user e-dorigatti commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189823961
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/21383#discussion_r189818845
--- Diff: python/pyspark/shuffle.py ---
@@ -67,6 +67,19 @@ def get_used_memory():
return 0
+def safe_iter(f):
+"""
GitHub user e-dorigatti opened a pull request:
https://github.com/apache/spark/pull/21383
[SPARK-23754][Python] Re-raising StopIteration in client code
## What changes were proposed in this pull request?
Make sure that `StopIteration`s raised in users' code do not silently
69 matches
Mail list logo
