[
https://issues.apache.org/jira/browse/OOZIE-3161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328127#comment-16328127
]
Satish Subhashrao Saley commented on OOZIE-3161:
------------------------------------------------
Committed to branch-4.3. Thank you Rohini for review.
> Spark job failing because of servlet-api-2.5 jar
> ------------------------------------------------
>
> Key: OOZIE-3161
> URL: https://issues.apache.org/jira/browse/OOZIE-3161
> Project: Oozie
> Issue Type: Bug
> Reporter: Satish Subhashrao Saley
> Assignee: Satish Subhashrao Saley
> Priority: Blocker
> Fix For: 4.3.1
>
> Attachments: OOZIE-3161-1.patch
>
>
> Spark Job fails with
> {code}
> 18/01/15 15:08:38 ERROR SparkContext: Error initializing SparkContext.
> java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s
> signer information does not match signer information of other classes in the
> same package
> at java.lang.ClassLoader.checkCerts(ClassLoader.java:898)
> at java.lang.ClassLoader.preDefineClass(ClassLoader.java:668)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:761)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
> at
> org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
> at
> org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
> at
> org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:126)
> at
> org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:113)
> at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:78)
> at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:62)
> at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:62)
> at
> scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
> at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
> at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:62)
> at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:63)
> at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:76)
> at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195)
> at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:473)
> at
> org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
> at org.apache.oozie.example.SparkFileCopy.main(SparkFileCopy.java:35)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542)
> {code}
> spark-streaming-flume-sink_2.10 is fetching jetty:servlet-api-2.5:jar. We
> need to exclude this.
> Here is dependency tree for sharelib/spark
> {code}
> [INFO] +- org.apache.spark:spark-streaming-flume_2.10:jar:1.6.1:compile
> [INFO] | +- org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.6.1:compile
> [INFO] | | +- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted for
> duplicate)
> [INFO] | | +- (org.apache.flume:flume-ng-core:jar:1.6.0:compile - omitted for
> duplicate)
> [INFO] | | +- (org.scala-lang:scala-library:jar:2.10.5:compile - omitted for
> duplicate)
> [INFO] | | \- (org.spark-project.spark:unused:jar:1.0.0:compile - omitted for
> duplicate)
> [INFO] | +- org.apache.flume:flume-ng-core:jar:1.6.0:compile
> [INFO] | | +- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted for
> duplicate)
> [INFO] | | +- org.apache.flume:flume-ng-configuration:jar:1.6.0:compile
> [INFO] | | | +- (org.slf4j:slf4j-api:jar:1.6.6:compile - version managed from
> 1.6.1; omitted for duplicate)
> [INFO] | | | +- (org.slf4j:slf4j-log4j12:jar:1.6.6:compile - version managed
> from 1.6.1; omitted for duplicate)
> [INFO] | | | \- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted
> for duplicate)
> [INFO] | | +- (org.slf4j:slf4j-api:jar:1.6.6:compile - version managed from
> 1.6.1; omitted for duplicate)
> [INFO] | | +- (commons-io:commons-io:jar:2.1:compile - version managed from
> 2.4; omitted for duplicate)
> [INFO] | | +- (commons-codec:commons-codec:jar:1.4:compile - version managed
> from 1.8; omitted for duplicate)
> [INFO] | | +- (log4j:log4j:jar:1.2.17:compile - version managed from 1.2.12;
> omitted for duplicate)
> [INFO] | | +- (org.slf4j:slf4j-log4j12:jar:1.6.6:compile - version managed
> from 1.6.1; omitted for duplicate)
> [INFO] | | +- (commons-cli:commons-cli:jar:1.2:compile - omitted for
> duplicate)
> [INFO] | | +- (commons-lang:commons-lang:jar:2.4:compile - version managed
> from 2.5; omitted for duplicate)
> [INFO] | | +- (org.apache.avro:avro:jar:1.7.4:compile - omitted for conflict
> with 1.7.7)
> [INFO] | | +- (org.apache.avro:avro-ipc:jar:1.7.4:compile - omitted for
> conflict with 1.7.7)
> [INFO] | | +- (joda-time:joda-time:jar:2.1:compile - version managed from
> 2.9; omitted for duplicate)
> [INFO] | | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
> [INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.14:compile (version managed
> from 6.1.26)
> [INFO] | | | +- (org.mortbay.jetty:jetty-util:jar:6.1.14:compile - omitted
> for conflict with 6.1.26)
> [INFO] | | | \- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
> [INFO] | | +- com.google.code.gson:gson:jar:2.7:compile (version managed from
> 2.2.2)
> {code}
> To reproduce:
> Build Oozie with spark-1 profile and deploy. Run the spark example (with
> yarn-master mode) which comes in Oozie examples.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
