Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21538#discussion_r194806008 --- Diff: python/pyspark/worker.py --- @@ -122,6 +123,10 @@ def read_single_udf(pickleSer, infile, eval_type): else: row_func = chain(row_func, f) + # make sure StopIteration's raised in the user code are not ignored + # when they are processed in a for loop, raise them as RuntimeError's instead + row_func = fail_on_stopiteration(row_func) --- End diff -- @e-dorigatti, I think it's fine to name it `func`. Let's reduce the diff so that other backports make less conflicts in the future.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org