GitHub user squito opened a pull request:
https://github.com/apache/spark/pull/19280
[SPARK-21928][CORE] Set classloader on SerializerManager private kryo
## What changes were proposed in this pull request?
We have to make sure thatthat SerializerManager's private instance of
kryo also uses the right classloader, regardless of the current thread
classloader. In particular, this fixes serde during remote cache
fetches, as those occur in netty threads.
## How was this patch tested?
Manual tests & existing suite via jenkins. I haven't been able to
reproduce this is in a unit test, because when a remote RDD partition can be
fetched, there is a warning message and then the partition is just recomputed
locally. I manually verified the warning message is no longer present.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/squito/spark SPARK-21928_ser_classloader
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19280.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19280
----
commit acbaf8b65629344d760360b768e89f1712af8942
Author: Imran Rashid <iras...@cloudera.com>
Date: 2017-09-19T15:19:43Z
[SPARK-21928][CORE] Set classloader on SerializerManager private kryo
We have to make sure thatthat SerializerManager's private instance of
kryo also uses the right classloader, regardless of the current thread
classloader. In particular, this fixes serde during remote cache
fetches, as those occur in netty threads.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
