[ https://issues.apache.org/jira/browse/SPARK-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13977897#comment-13977897 ]
Patrick Wendell commented on SPARK-1480: ---------------------------------------- https://github.com/apache/spark/pull/398/files > Choose classloader consistently inside of Spark codebase > -------------------------------------------------------- > > Key: SPARK-1480 > URL: https://issues.apache.org/jira/browse/SPARK-1480 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Reporter: Patrick Wendell > Assignee: Patrick Wendell > Priority: Blocker > Fix For: 1.0.0 > > > The Spark codebase is not always consistent on which class loader it uses > when classlaoders are explicitly passed to things like serializers. This > caused SPARK-1403 and also causes a bug where when the driver has a modified > context class loader it is not translated correctly in local mode to the > (local) executor. > In most cases what we want is the following behavior: > 1. If there is a context classloader on the thread, use that. > 2. Otherwise use the classloader that loaded Spark. > We should just have a utility function for this and call that function > whenever we need to get a classloader. > Note that SPARK-1403 is a workaround for this exact problem (it sets the > context class loader because downstream code assumes it is set). Once this > gets fixed in a more general way SPARK-1403 can be reverted. -- This message was sent by Atlassian JIRA (v6.2#6252)