Could you put the implicit def in an object? That should work, as objects are never serialized.
On Wed, Apr 29, 2015 at 6:28 PM, [email protected] < [email protected]> wrote: > Thank you for your pointers , it`s very helpful to me , in this scenario > how can i use the implicit def in the enclosing class ? > > > > *From:* Tathagata Das <[email protected]> > *Date:* 2015-04-30 07:00 > *To:* [email protected] > *CC:* user <[email protected]> > *Subject:* Re: implicit function in SparkStreaming > I believe that the implicit def is pulling in the enclosing class (in > which the def is defined) in the closure which is not serializable. > > > On Wed, Apr 29, 2015 at 4:20 AM, [email protected] < > [email protected]> wrote: > >> Hi guys, >> I`m puzzled why i cant use the implicit function in spark streaming to >> cause Task not serializable . >> >> code snippet: >> >> implicit final def str2KeyValue(s:String): (String,String) = { >> val message = s.split("\\|") >> if(message.length >= 2) >> (message(0),message(1)) >> else if(message.length == 1) { >> (message(0), "") >> } >> else >> ("","") >> } >> >> def filter(stream:DStream[String]) :DStream[String] = { >> stream.filter(s => { >> (s._1=="Action" && s._2=="TRUE") >> }) >> >> >> Could you please give me some pointers ? Thank you . >> >> ------------------------------ >> [email protected] >> > >
