I'm new to spark programming and here I'm trying to use third party class in
map with kryo serializer
val deviceApi = new DeviceApi()
deviceApi.loadDataFromStream(this.getClass.getClassLoader.getResourceAsStream("20140730.json"))
val properties = uaRDD1.map(line => deviceApi.getProperties(line)
Here is kryo register code
import mobi.mtld.da.device.DeviceApi
class MyRegistrator extends KryoRegistrator {
override def registerClasses(kryo: Kryo) {
kryo.register(classOf[DeviceApi])
}
}
I'm still seeing this error
Caused by: org.apache.spark.SparkException: Job aborted: Task not
serializable: java.io.NotSerializableException:
mobi.mtld.da.device.DeviceApi
Can anyone please let me know if something is wrong
Thanks
Giri
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Issue-using-kryo-serilization-tp11129.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
