I created a JIRA ticket to track this: https://issues.apache.org/jira/browse/SPARK-2928
Let me know if you need help with it. On Fri, Aug 8, 2014 at 10:40 AM, Reynold Xin <r...@databricks.com> wrote: > Yes, I'm pretty sure it doesn't actually use the right serializer in > TorrentBroadcast: > https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L232 > > And TorrentBroadcast is turned on by default for 1.1 right now. Do you > want to submit a pull request to fix that? This would be a critical fix for > 1.1 that's worth doing. > > > > On Fri, Aug 8, 2014 at 10:37 AM, Ron Gonzalez <zlgonza...@yahoo.com> > wrote: > >> Oops, exception is below. >> >> For local, it works and that's the case since TorrentBroadcast has if >> !isLocal, then that's the only time the broadcast actually happens. It >> really seems as if the Kryo wrapper didn't kick in for some reason. Do we >> have a unit test that tests the Kryo serialization that I can give a try? >> >> Thanks, >> >> Ron >> >> >> Exception in thread "Driver" java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:180) >> Caused by: java.io.NotSerializableException: >> org.apache.avro.generic.GenericData$Record >> - custom writeObject data (class "scala.collection.mutable.HashMap") >> >> >> >> On Friday, August 8, 2014 10:16 AM, Reynold Xin <r...@databricks.com> >> wrote: >> >> >> Looks like you didn't actually paste the exception message. Do you mind >> doing that? >> >> >> >> On Fri, Aug 8, 2014 at 10:14 AM, Reynold Xin <r...@databricks.com> wrote: >> >> > Pasting a better formatted trace: >> > >> > >> > >> > at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180) >> > at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) >> > at >> > >> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:137) >> > at >> > >> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:135) >> > at >> > >> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) >> > at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) >> > at >> > >> scala.collection.mutable.HashTable$class.serializeTo(HashTable.scala:124) >> > at scala.collection.mutable.HashMap.serializeTo(HashMap.scala:39) >> > at scala.collection.mutable.HashMap.writeObject(HashMap.scala:135) >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:606) at >> > java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) >> > at >> > java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) >> > at >> > >> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) >> > at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) >> > at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) >> > at org.apache.spark.util.Utils$.serialize(Utils.scala:64) >> > at >> > >> org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:232) >> > at >> > >> org.apache.spark.broadcast.TorrentBroadcast.sendBroadcast(TorrentBroadcast.scala:85) >> > at >> > >> org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:66) >> > at >> > >> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:36) >> > at >> > >> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:29) >> > at >> > >> > >> org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62) >> > at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809) >> > >> > >> > On Fri, Aug 8, 2014 at 10:12 AM, Ron Gonzalez < >> > zlgonza...@yahoo.com.invalid> wrote: >> > >> >> Hi, >> >> I have a running spark app against the released version of 1.0.1. I >> >> recently decided to try and upgrade to the trunk version. Interestingly >> >> enough, after building the 1.1.0-SNAPSHOT assembly, replacing it as my >> >> assembly in my app caused errors. In particular, it seems Kryo >> >> serialization isn't taking. Replacing it with 1.0.1 automatically gets >> it >> >> working again. >> >> >> >> Any thoughts? Is this a known issue? >> >> >> >> Thanks, >> >> Ron >> >> >> >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180) >> >> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) >> at >> >> >> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:137) >> >> at >> >> >> scala.collection.mutable.HashMap$$anonfun$writeObject$1.apply(HashMap.scala:135) >> >> at >> >> >> scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) >> >> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) at >> >> >> scala.collection.mutable.HashTable$class.serializeTo(HashTable.scala:124) >> >> at scala.collection.mutable.HashMap.serializeTo(HashMap.scala:39) at >> >> scala.collection.mutable.HashMap.writeObject(HashMap.scala:135) at >> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> at >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:606) at >> >> >> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988) >> >> at >> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495) >> >> at >> >> >> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431) >> >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at >> >> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at >> >> org.apache.spark.util.Utils$.serialize(Utils.scala:64) at >> >> >> org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:232) >> >> at >> >> >> org.apache.spark.broadcast.TorrentBroadcast.sendBroadcast(TorrentBroadcast.scala:85) >> >> at >> >> >> org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:66) >> >> at >> >> >> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:36) >> >> at >> >> >> org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:29) >> >> at >> >> >> org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62) >> >> at org.apache.spark.SparkContext.broadcast(SparkContext.scala:809) >> > >> > >> > >> >> >> >
