Satish Subhashrao Saley created OOZIE-3161:
----------------------------------------------
Summary: Spark job failing because of servlet-api-2.5 jar
Key: OOZIE-3161
URL: https://issues.apache.org/jira/browse/OOZIE-3161
Project: Oozie
Issue Type: Bug
Environment: Spark Job fails with
{code}
18/01/15 15:08:38 ERROR SparkContext: Error initializing SparkContext.
java.lang.SecurityException: class "javax.servlet.FilterRegistration"'s signer
information does not match signer information of other classes in the same
package
at java.lang.ClassLoader.checkCerts(ClassLoader.java:898)
at java.lang.ClassLoader.preDefineClass(ClassLoader.java:668)
at java.lang.ClassLoader.defineClass(ClassLoader.java:761)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at
org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:136)
at
org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:129)
at
org.spark-project.jetty.servlet.ServletContextHandler.<init>(ServletContextHandler.java:98)
at
org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:126)
at
org.apache.spark.ui.JettyUtils$.createServletHandler(JettyUtils.scala:113)
at org.apache.spark.ui.WebUI.attachPage(WebUI.scala:78)
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:62)
at org.apache.spark.ui.WebUI$$anonfun$attachTab$1.apply(WebUI.scala:62)
at
scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at org.apache.spark.ui.WebUI.attachTab(WebUI.scala:62)
at org.apache.spark.ui.SparkUI.initialize(SparkUI.scala:63)
at org.apache.spark.ui.SparkUI.<init>(SparkUI.scala:76)
at org.apache.spark.ui.SparkUI$.create(SparkUI.scala:195)
at org.apache.spark.ui.SparkUI$.createLiveUI(SparkUI.scala:146)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:473)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
at org.apache.oozie.example.SparkFileCopy.main(SparkFileCopy.java:35)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542)
{code}
spark-streaming-flume-sink_2.10 is fetching jetty:servlet-api-2.5:jar. We need
to exclude this.
Here is dependency tree for sharelib/spark
{code}
[INFO] +- org.apache.spark:spark-streaming-flume_2.10:jar:1.6.1:compile
[INFO] | +- org.apache.spark:spark-streaming-flume-sink_2.10:jar:1.6.1:compile
[INFO] | | +- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted for
duplicate)
[INFO] | | +- (org.apache.flume:flume-ng-core:jar:1.6.0:compile - omitted for
duplicate)
[INFO] | | +- (org.scala-lang:scala-library:jar:2.10.5:compile - omitted for
duplicate)
[INFO] | | \- (org.spark-project.spark:unused:jar:1.0.0:compile - omitted for
duplicate)
[INFO] | +- org.apache.flume:flume-ng-core:jar:1.6.0:compile
[INFO] | | +- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted for
duplicate)
[INFO] | | +- org.apache.flume:flume-ng-configuration:jar:1.6.0:compile
[INFO] | | | +- (org.slf4j:slf4j-api:jar:1.6.6:compile - version managed from
1.6.1; omitted for duplicate)
[INFO] | | | +- (org.slf4j:slf4j-log4j12:jar:1.6.6:compile - version managed
from 1.6.1; omitted for duplicate)
[INFO] | | | \- (org.apache.flume:flume-ng-sdk:jar:1.6.0:compile - omitted for
duplicate)
[INFO] | | +- (org.slf4j:slf4j-api:jar:1.6.6:compile - version managed from
1.6.1; omitted for duplicate)
[INFO] | | +- (commons-io:commons-io:jar:2.1:compile - version managed from
2.4; omitted for duplicate)
[INFO] | | +- (commons-codec:commons-codec:jar:1.4:compile - version managed
from 1.8; omitted for duplicate)
[INFO] | | +- (log4j:log4j:jar:1.2.17:compile - version managed from 1.2.12;
omitted for duplicate)
[INFO] | | +- (org.slf4j:slf4j-log4j12:jar:1.6.6:compile - version managed from
1.6.1; omitted for duplicate)
[INFO] | | +- (commons-cli:commons-cli:jar:1.2:compile - omitted for duplicate)
[INFO] | | +- (commons-lang:commons-lang:jar:2.4:compile - version managed from
2.5; omitted for duplicate)
[INFO] | | +- (org.apache.avro:avro:jar:1.7.4:compile - omitted for conflict
with 1.7.7)
[INFO] | | +- (org.apache.avro:avro-ipc:jar:1.7.4:compile - omitted for
conflict with 1.7.7)
[INFO] | | +- (joda-time:joda-time:jar:2.1:compile - version managed from 2.9;
omitted for duplicate)
[INFO] | | +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] | | +- org.mortbay.jetty:jetty:jar:6.1.14:compile (version managed from
6.1.26)
[INFO] | | | +- (org.mortbay.jetty:jetty-util:jar:6.1.14:compile - omitted for
conflict with 6.1.26)
[INFO] | | | \- org.mortbay.jetty:servlet-api-2.5:jar:6.1.14:compile
[INFO] | | +- com.google.code.gson:gson:jar:2.7:compile (version managed from
2.2.2)
{code}
To reproduce:
Build Oozie with spark-1 profile and deploy. Run the spark example (with
yarn-master mode) which comes in Oozie examples.
Reporter: Satish Subhashrao Saley
Assignee: Satish Subhashrao Saley
Fix For: 4.3.1
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
