inpefess commented on a change in pull request #20691: [SPARK-18161] [Python] Update cloudpickle to v0.6.1 URL: https://github.com/apache/spark/pull/20691#discussion_r249238272
########## File path: python/pyspark/broadcast.py ########## @@ -110,7 +110,7 @@ def __init__(self, sc=None, value=None, pickle_registry=None, path=None, def dump(self, value, f): try: - pickle.dump(value, f, 2) + pickle.dump(value, f, pickle.HIGHEST_PROTOCOL) Review comment: It happened here: https://github.com/apache/spark/commit/6cf507685efd01df77d663145ae08e48c7f92948#diff-bb67501acde415576c589b478e16c60aR82 Since then it never changed. I agree that there was no particular reason for that since pickle.HIGHEST_PROTOCOL in Python 2 versions is 2 for ages, not 3 or 4. Using pickle.HIGHEST_PROTOCOL consistently should be safe for that reason. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org