inpefess commented on a change in pull request #20691: [SPARK-18161] [Python]
Update cloudpickle to v0.6.1
URL: https://github.com/apache/spark/pull/20691#discussion_r249238272
##########
File path: python/pyspark/broadcast.py
##########
@@ -110,7 +110,7 @@ def __init__(self, sc=None, value=None,
pickle_registry=None, path=None,
def dump(self, value, f):
try:
- pickle.dump(value, f, 2)
+ pickle.dump(value, f, pickle.HIGHEST_PROTOCOL)
Review comment:
It happened here:
https://github.com/apache/spark/commit/6cf507685efd01df77d663145ae08e48c7f92948#diff-bb67501acde415576c589b478e16c60aR82
Since then it never changed.
I agree that there was no particular reason for that since
pickle.HIGHEST_PROTOCOL in Python 2 versions is 2 for ages, not 3 or 4. Using
pickle.HIGHEST_PROTOCOL consistently should be safe for that reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
