I believe that the implicit def is pulling in the enclosing class (in which
the def is defined) in the closure which is not serializable.
On Wed, Apr 29, 2015 at 4:20 AM, guoqing0...@yahoo.com.hk <
guoqing0...@yahoo.com.hk> wrote:
> Hi guys,
> I`m puzzled why i cant use the implicit function in spark streaming to
> cause Task not serializable .
>
> code snippet:
>
> implicit final def str2KeyValue(s:String): (String,String) = {
> val message = s.split("\\|")
> if(message.length >= 2)
> (message(0),message(1))
> else if(message.length == 1) {
> (message(0), "")
> }
> else
> ("","")
> }
>
> def filter(stream:DStream[String]) :DStream[String] = {
> stream.filter(s => {
> (s._1=="Action" && s._2=="TRUE")
> })
>
>
> Could you please give me some pointers ? Thank you .
>
> ------------------------------
> guoqing0...@yahoo.com.hk
>
