Github user coolfrood commented on the pull request:
https://github.com/apache/spark/pull/6361#issuecomment-104786749
@JoshRosen, thanks for your comments. I'll work on these. Even without
the tests, I have identified two places where Kryo closure serialization
doesn't work right now:
1. `HadoopPartition` serialization when working with `sc.textFile`
2. Accumulator values don't get serialized correctly (or at all) on the
executors.
There's clearly quite a bit more work required. I think in most cases,
it's due to custom serialization/deserialization code written for Java
serialization, which doesn't get invoked when using Kryo.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
