Re: Getting around Serializability issues for types not in my control

2015-03-23 Thread Adelbert Chang
Is there no way to pull out the bits of the instance I want before I sent it through the closure for aggregate? I did try pulling things out, along the lines of def foo[G[_], B](blah: Blah)(implicit G: Applicative[G]) = { val lift: B = G[RDD[B]] = b = G.point(sparkContext.parallelize(List(b)))

Re: Getting around Serializability issues for types not in my control

2015-03-23 Thread Adelbert Chang
, Cody Koeninger c...@koeninger.org wrote: Have you tried instantiating the instance inside the closure, rather than outside of it? If that works, you may need to switch to use mapPartition / foreachPartition for efficiency reasons. On Mon, Mar 23, 2015 at 3:03 PM, Adelbert Chang adelbe