[
https://issues.apache.org/jira/browse/SPARK-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13987454#comment-13987454
]
Sean Owen commented on SPARK-1693:
----------------------------------
[~pwendell] The Hadoop artifacts want version 2.5, as you see. The Servlet
3.0.x *APIs* should be a superset of 2.5. In theory clients implementing or
using 2.5 will be fine with it, which means there's an 80% chance it's actually
true. Hence I think a solution is to enforce use of 3.0.1, and exclude all
other copies of the servlet API.
Normally that's as simple as declaring a runtime dependency on the latest
version of the artifact, which will override any transitive dependencies. The
problem here is exactly because that's not true; Eclipse has its own artifact
for the servlet 3.0 APIs.
And [~witgo], as you noted, the artifact name actually changed from 2.5 to
3.0.1! So depending on it will *not* remove Hadoop's dependency. I think one
more change is needed, to exclude Hadoop's dependency on the old
javax.servlet:servlet-api artifact.
Basically, "mvn ... dependency:tree | grep javax.servlet" should only show the
new 3.0.1 dependency.
This should hopefully solve the client-side, unit test failures at least, which
is necessary. [~witgo] I assume you have found it makes the tests pass? I do
not think it will affect compatibility with a cluster, as this isn't changing
or affecting how the client talks to a cluster.
> Dependent on multiple versions of servlet-api jars lead to throw an
> SecurityException when Spark built for hadoop 2.3.0 , 2.4.0
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-1693
> URL: https://issues.apache.org/jira/browse/SPARK-1693
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Guoqiang Li
> Assignee: Guoqiang Li
> Priority: Blocker
> Fix For: 1.0.0
>
> Attachments: log.txt
>
>
> {code}mvn test -Pyarn -Dhadoop.version=2.4.0 -Dyarn.version=2.4.0 >
> log.txt{code}
> The log:
> {code}
> UnpersistSuite:
> - unpersist RDD *** FAILED ***
> java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s
> signer information does not match signer information of other classes in the
> same package
> at java.lang.ClassLoader.checkCerts(ClassLoader.java:952)
> at java.lang.ClassLoader.preDefineClass(ClassLoader.java:666)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:794)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
