Here is the relevant snippet of code in my main program:
===========================================
sparkConf.set("spark.serializer",
"org.apache.spark.serializer.KryoSerializer")
sparkConf.set("spark.kryo.registrationRequired", "true")
val summaryDataClass = classOf[SummaryData]
val summaryViewClass = classOf[SummaryView]
sparkConf.registerKryoClasses(Array(
summaryDataClass, summaryViewClass))
===========================================
I get the following error:
Exception in thread "main" java.lang.reflect.InvocationTargetException
...
Caused by: org.apache.spark.SparkException: Failed to load class to
register with Kryo
...
Caused by: java.lang.ClassNotFoundException:
com.dtex.analysis.transform.SummaryData
Note that the class in question SummaryData is in the same package as the
main program and hence in the same jar.
What do I need to do to make this work?
Thanks,
arun
