PS I think that solving "not serializable" exceptions by adding 'transient' is usually a mistake. It's a band-aid on a design problem.
transient causes the default serialization mechanism to not serialize the field when the object is serialized. When deserialized, this field will be null, which often compromises the class's assumptions about state. This keyword is only appropriate when the field can safely be recreated at any time -- things like cached values. In Java, this commonly comes up when declaring anonymous (therefore non-static) inner classes, which have an invisible reference to the containing instance, which can easily cause it to serialize the enclosing class when it's not necessary at all. Inner classes should be static in this case, if possible. Passing values as constructor params takes more code but let you tightly control what the function references. On Wed, Aug 13, 2014 at 2:47 PM, Jaideep Dhok <jaideep.d...@inmobi.com> wrote: > Hi, > I have faced a similar issue when trying to run a map function with predict. > In my case I had some non-serializable fields in my calling class. After > making those fields transient, the error went away. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
