[
https://issues.apache.org/jira/browse/SPARK-11093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-11093:
------------------------------------
Assignee: (was: Apache Spark)
> ChildFirstURLClassLoader#getResources should return all found resources, not
> just those in the child classloader
> ----------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-11093
> URL: https://issues.apache.org/jira/browse/SPARK-11093
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.5.1
> Reporter: Adam Lewandowski
>
> Currently when using a child-first classloader
> (spark.{driver|executor}.userClassPathFirst = true), the getResources method
> does not return any matching resources from the parent classloader if the
> child classloader contains any. This is not child-first, it's child-only and
> is inconsistent with how the default parent-first classloaders work in the
> JDK (all found resources are returned from both classloaders). It is also
> inconsistent with how child-first classloaders work in other environments
> (Servlet containers, for example).
> ChildFirstURLClassLoader#getResources() should return resources found from
> both the child and the parent classloaders, placing any found from the child
> classloader first.
> For reference, the specific use case where I encountered this problem was
> running Spark on AWS EMR in a child-first arrangement (due to guava version
> conflicts), where Akka's configuration file (reference.conf) was made
> available in the parent classloader, but was not visible to the Typesafe
> config library which uses Classloader.getResources() on the Thread's context
> classloader to find them. This resulted in a fatal error from the Config
> library: "com.typesafe.config.ConfigException$Missing: No configuration
> setting found for key 'akka.version'" .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
