ClassNotFound exception from closure

[Yana Kadiyska]

Hi folks,

running into a pretty strange issue -- I have a ClassNotFound exception
from a closure?! My code looks like this:

 val jRdd1 = table.map(cassRow=>{
      val lst = List(cassRow.get[Option[Any]](0),cassRow.get[Option[Any]](1))
      Row.fromSeq(lst)
    })
    println(s"This one worked ..."+jRdd1.first.toString())

    println("SILLY -----------------------------------")
    val sillyRDD=sc.parallelize(1 to 100)
    val jRdd2 = sillyRDD.map(value=>{
      val cols = (0 to 2).map(i=>"foo").toList //3 foos per row
      println(s"Valus "+cols.mkString("|"))
      Row.fromSeq(cols)
    })
    println(s"This one worked too "+jRdd2.first.toString())

​
and the exception I see goes:

This one worked ...[Some(1234),Some(1434123162)]
SILLY -----------------------------------
Exception in thread "main" java.lang.ClassNotFoundException:
HardSparkJob$anonfun$3$anonfun$4
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:270)
        at 
org.apache.spark.util.InnerClosureFinder$anon$4.visitMethodInsn(ClosureCleaner.scala:455)
        at 
com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassReader.accept(Unknown
Source)
        at 
com.esotericsoftware.reflectasm.shaded.org.objectweb.asm.ClassReader.accept(Unknown
Source)
        at 
org.apache.spark.util.ClosureCleaner$.getInnerClosureClasses(ClosureCleaner.scala:101)
        at 
org.apache.spark.util.ClosureCleaner$.org$apache$spark$util$ClosureCleaner$clean(ClosureCleaner.scala:197)
        at org.apache.spark.util.ClosureCleaner$.clean(ClosureCleaner.scala:132)
        at org.apache.spark.SparkContext.clean(SparkContext.scala:1891)
        at org.apache.spark.rdd.RDD$anonfun$map$1.apply(RDD.scala:294)
        at org.apache.spark.rdd.RDD$anonfun$map$1.apply(RDD.scala:293)
        at 
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:148)
        at 
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:109)
        at org.apache.spark.rdd.RDD.withScope(RDD.scala:286)
        at org.apache.spark.rdd.RDD.map(RDD.scala:293)
        at HardSparkJob$.testUnionViaRDD(SparkTest.scala:127)
        at HardSparkJob$.main(SparkTest.scala:104)
        at HardSparkJob.main(SparkTest.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$runMain(SparkSubmit.scala:664)
        at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

​

I don't quite know what to make of this error. The stacktrace shows a
problem with my code at sillyRDD.map(SparkTest.scala:127)

I'm running Spark 1.4 CDH prebuilt with

bin/spark-submit --class HardSparkJob --master mesos://$MESOS_MASTER
../MyJar.jar

Any insight much appreciated