I have to use .lines.toArray.toSeq A little tricky.
[image: --] Xi Shen [image: http://]about.me/davidshen <http://about.me/davidshen?promo=email_sig> <http://about.me/davidshen> On Fri, Mar 27, 2015 at 4:41 PM, Xi Shen <davidshe...@gmail.com> wrote: > Hi, > > I want to load my data in this way: > > sc.wholeTextFiles(opt.input) map { x => (x._1, > x._2.lines.filter(!_.isEmpty).toSeq) } > > > But I got > > java.io.NotSerializableException: scala.collection.Iterator$$anon$13 > > But if I use "x._2.split('\n')", I can get the expected result. I want to > know what's wrong with using the "lines()" function. > > > Thanks, > > [image: --] > Xi Shen > [image: http://]about.me/davidshen > <http://about.me/davidshen?promo=email_sig> > <http://about.me/davidshen> >
