Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/21157
Is it possible to keep the current hack for things which can't be pickled,
but remove the hack in the situation where the namedtuple is well behaved and
it could be pickled directly by cloudpickle? That way we don't have a
functionality regression but we also improve handling of named tuples more
generally. Even if so, it would probably be best to wait for 3.0 since this is
a pretty core change in terms of PySpark.
Before you put in the work though let's see if that the consensus approach
(if possible).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
