Hi,
I have maven based mixed scala/java application that can submit spar jobs.
My application jar "myapp.jar" has some nested jars inside lib folder. It's
a fat jar created using spring-boot-maven plugin which nest other jars
inside lib folder of parent jar. I prefer not to create shaded flat jar as
that would create other issues.
one of the nested jar is "common.jar". I have defined class-path attribute
in Manifest file like Class-Path: lib/common.jar. Spark executor throws
java.lang.NoClassDefFoundError:com/myapp/common/myclass error when
submitting application in yarn-client mode.
Class(com/myapp/common/myclass.class) and jar(common.jar) is there and
nested inside my main myapp.jar.
Does spark executor jvm can load nested jars here?
spark (jvm classloader) can find all the classes those are flat inside
myapp.jar itself. i.e. com/myapp/abc.class, com/myapp/xyz.class etc.
spark executor classloader can also find some classes from nested jar but
it throws NoClassDefFoundError some other classes in same nested jar!
here's the error:
Caused by: org.apache.spark.SparkException: Job aborted due to stage
failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost
task 0.3 in stage 0.0 (TID 3, host4.local):
java.lang.NoClassDefFoundError: com/myapp/common/myclass
at com.myapp.UserProfileRDD$.parse(UserProfileRDDInit.scala:111)
at
com.myapp.UserProfileRDDInit$$anonfun$generateUserProfileRDD$1.apply(UserProfileRDDInit.scala:87)
at
com.myapp.UserProfileRDDInit$$anonfun$generateUserProfileRDD$1.applyUserProfileRDDInit.scala:87)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at org.apache.spark.storage.MemoryStore.unrollSafely(MemoryStore.scala:249)
at org.apache.spark.CacheManager.putInBlockManager(CacheManager.scala:172)
at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:79)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:242)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
at org.apache.spark.scheduler.Task.run(Task.scala:64)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)Caused by:
java.lang.ClassNotFoundException:
com.myapp.common.myclass
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 14 more
I do submit myapp.jar with sparkConf.setJar(String[] {"myapp.jar"}) and
also tried setting it onspark.yarn.executor.extraClassPath
As a workaround, I extracted myapp.jar and set sparkConf.setJar(String[]
{"myapp.jar","lib/common.jar"}) manually and error went away but obviously
I have to do that for all the nested jar which is not desirable.
Thanks
--
[image: What's New with Xactly] <http://www.xactlycorp.com/email-click/>
<https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn]
<https://www.linkedin.com/company/xactly-corporation> [image: Twitter]
<https://twitter.com/Xactly> [image: Facebook]
<https://www.facebook.com/XactlyCorp> [image: YouTube]
<http://www.youtube.com/xactlycorporation>
