I'm not sure if this is a Hadoop-centric issue or not. I had similar issues with non-serializable external library classes.
I used a Kryo config (as illustrated here <https://spark.apache.org/docs/latest/tuning.html#data-serialization> ) and registered the one troublesome class. It seemed to work after that. Here's a link to the thread <http://apache-spark-user-list.1001560.n3.nabble.com/Un-serializable-3rd-party-classes-Spark-Java-td7815.html> I asked on. Take a look at the other solutions proposed as well. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Serialization-problem-in-Spark-tp7049p7975.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
