In my spark job, I have a loop something like this: bla.forEachRdd(rdd => { //init some vars rdd.forEachPartition(partiton => { //init some vars partition.foreach(kv => { ...
I am seeing serialization errors (unread block data), because I think spark is trying to serialize the whole containing class. But I have been careful not to reference instance vars in the block. Is there a way to see exactly what class is failing serialization, and maybe how spark decided it needs to be serialized? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-trace-debug-serialization-tp18230.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org