SparkContext.wholeTextFiles throws not serializable exception

Hi, I want to load my data in this way: sc.wholeTextFiles(opt.input) map { x = (x._1, x._2.lines.filter(!_.isEmpty).toSeq) } But I got java.io.NotSerializableException: scala.collection.Iterator$$anon$13 But if I use x._2.split('\n'), I can get the expected result. I want to know what's

Re: SparkContext.wholeTextFiles throws not serializable exception

I have to use .lines.toArray.toSeq A little tricky. [image: --] Xi Shen [image: http://]about.me/davidshen http://about.me/davidshen?promo=email_sig http://about.me/davidshen On Fri, Mar 27, 2015 at 4:41 PM, Xi Shen davidshe...@gmail.com wrote: Hi, I want to load my data in this way: