Bryan Cutler created SPARK-23159:
------------------------------------
Summary: Update Cloudpickle to match version 0.4.2
Key: SPARK-23159
URL: https://issues.apache.org/jira/browse/SPARK-23159
Project: Spark
Issue Type: Improvement
Components: PySpark
Affects Versions: 2.3.0
Reporter: Bryan Cutler
Update PySpark's version of Cloudpickle to match version 0.4.2. The reasons
for doing this are:
* Pick up bug fixes, improvements with newer version
* Match a specific version as close as possible (Spark has additional changes
that might be necessary) to make future upgrades easier
There are newer versions of Cloudpickle that can fix bugs with NamedTuple
pickling (that Spark currently has workarounds for), but these include other
changes that need some verification before bringing into Spark. Upgrading
first to 0.4.2 will help make this verification easier.
Discussion on the mailing list:
http://apache-spark-developers-list.1001551.n3.nabble.com/Thoughts-on-Cloudpickle-Update-td23188.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
