[
https://issues.apache.org/jira/browse/OOZIE-2071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279167#comment-14279167
]
Robert Kanter commented on OOZIE-2071:
--------------------------------------
I'm still getting this exception:
{noformat}
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain],
main() threw exception,
org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder;
java.lang.NoSuchMethodError:
org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder;
at org.apache.spark.Logging$class.initializeLogging(Logging.scala:116)
at
org.apache.spark.Logging$class.initializeIfNecessary(Logging.scala:107)
at org.apache.spark.Logging$class.log(Logging.scala:51)
at org.apache.spark.SecurityManager.log(SecurityManager.scala:143)
at org.apache.spark.Logging$class.logInfo(Logging.scala:59)
at org.apache.spark.SecurityManager.logInfo(SecurityManager.scala:143)
at
org.apache.spark.SecurityManager.setViewAcls(SecurityManager.scala:208)
at org.apache.spark.SecurityManager.<init>(SecurityManager.scala:167)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:151)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:203)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:53)
at org.apache.spark.examples.JavaWordCount.main(JavaWordCount.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:105)
at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:96)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:44)
at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:228)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
A quick google search says something about having different versions of the
slf4j jars in the classpath. I see these coming from Hadoop:
{noformat}
/Users/rkanter/dev/hadoop-1.2.0/libexec/../lib/slf4j-api-1.4.3.jar
/Users/rkanter/dev/hadoop-1.2.0/libexec/../lib/slf4j-log4j12-1.4.3.jar
{noformat}
And these coming from the Spark sharelib:
{noformat}
/Users/rkanter/dev/hadoop/dirs/mapred/taskTracker/distcache/-8420537941038394191_-606177215_1789737059/localhost/user/rkanter/share/lib/lib_20150114145949/spark/slf4j-api-1.6.6.jar
/Users/rkanter/dev/hadoop/dirs/mapred/taskTracker/distcache/-2382169348551914454_189755936_1789737068/localhost/user/rkanter/share/lib/lib_20150114145949/spark/slf4j-log4j12-1.6.6.jar
{noformat}
I tried removing the slf4j jars from the spark sharelib, but still got the same
error. Perhaps we need different Spark artifacts for Hadoop 1.x?
> Add a Spark example
> -------------------
>
> Key: OOZIE-2071
> URL: https://issues.apache.org/jira/browse/OOZIE-2071
> Project: Oozie
> Issue Type: Task
> Components: examples
> Affects Versions: trunk
> Reporter: Robert Kanter
> Assignee: pavan kumar kolamuri
> Fix For: trunk
>
> Attachments: OOZIE-2071-v1.patch, OOZIE-2071.patch
>
>
> With the new Spark action, we should have a Spark example workflow that runs
> some Spark job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
