+1 what Sean said. And if there are too many state/argument parameters for your taste, you can always create a dedicated (serializable) class to encapsulate them.
Sent while mobile. Pls excuse typos etc. On Aug 13, 2014 6:58 AM, "Sean Owen" <so...@cloudera.com> wrote: > PS I think that solving "not serializable" exceptions by adding > 'transient' is usually a mistake. It's a band-aid on a design problem. > > transient causes the default serialization mechanism to not serialize > the field when the object is serialized. When deserialized, this field > will be null, which often compromises the class's assumptions about > state. This keyword is only appropriate when the field can safely be > recreated at any time -- things like cached values. > > In Java, this commonly comes up when declaring anonymous (therefore > non-static) inner classes, which have an invisible reference to the > containing instance, which can easily cause it to serialize the > enclosing class when it's not necessary at all. > > Inner classes should be static in this case, if possible. Passing > values as constructor params takes more code but let you tightly > control what the function references. > > On Wed, Aug 13, 2014 at 2:47 PM, Jaideep Dhok <jaideep.d...@inmobi.com> > wrote: > > Hi, > > I have faced a similar issue when trying to run a map function with > predict. > > In my case I had some non-serializable fields in my calling class. After > > making those fields transient, the error went away. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
