If you run the JVM with -Dsun.io.serialization.extendedDebugInfo=true it will dump the 'path' to the unserializable object when an error occurs.
On Wed, Nov 5, 2014 at 11:56 PM, ankits <[email protected]> wrote: > In my spark job, I have a loop something like this: > > bla.forEachRdd(rdd => { > //init some vars > rdd.forEachPartition(partiton => { > //init some vars > partition.foreach(kv => { > ... > > I am seeing serialization errors (unread block data), because I think spark > is trying to serialize the whole containing class. But I have been careful > not to reference instance vars in the block. > > Is there a way to see exactly what class is failing serialization, and maybe > how spark decided it needs to be serialized? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-trace-debug-serialization-tp18230.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
