The runtime attempts to serialize everything required by records, and also any lambdas/closures you use. Small, simple types are less likely to run into this problem.
Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) Typesafe <http://typesafe.com> @deanwampler <http://twitter.com/deanwampler> http://polyglotprogramming.com On Sun, Apr 5, 2015 at 4:21 PM, Jeetendra Gangele <gangele...@gmail.com> wrote: > You are right I have class called VendorRecord which is not serializable > also this class object have many sub classed(may be 30 or more).Do I need > to recursively serialize all? > > > > On 4 April 2015 at 18:14, Dean Wampler <deanwamp...@gmail.com> wrote: > >> Without the rest of your code, it's hard to know what might be >> unserializable. >> >> Dean Wampler, Ph.D. >> Author: Programming Scala, 2nd Edition >> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >> Typesafe <http://typesafe.com> >> @deanwampler <http://twitter.com/deanwampler> >> http://polyglotprogramming.com >> >> On Sat, Apr 4, 2015 at 7:56 AM, Jeetendra Gangele <gangele...@gmail.com> >> wrote: >> >>> >>> Hi I have tried with parallelize but i got the below exception >>> >>> java.io.NotSerializableException: pacific.dr.VendorRecord >>> >>> Here is my code >>> >>> List<VendorRecord> >>> vendorRecords=blockingKeys.getMatchingRecordsWithscan(matchKeysOutput); >>> JavaRDD<VendorRecord> lines = sc.parallelize(vendorRecords) >>> >>> >>> On 2 April 2015 at 21:11, Dean Wampler <deanwamp...@gmail.com> wrote: >>> >>>> Use JavaSparkContext.parallelize. >>>> >>>> >>>> http://spark.apache.org/docs/latest/api/java/org/apache/spark/api/java/JavaSparkContext.html#parallelize(java.util.List) >>>> >>>> Dean Wampler, Ph.D. >>>> Author: Programming Scala, 2nd Edition >>>> <http://shop.oreilly.com/product/0636920033073.do> (O'Reilly) >>>> Typesafe <http://typesafe.com> >>>> @deanwampler <http://twitter.com/deanwampler> >>>> http://polyglotprogramming.com >>>> >>>> On Thu, Apr 2, 2015 at 11:33 AM, Jeetendra Gangele < >>>> gangele...@gmail.com> wrote: >>>> >>>>> Hi All >>>>> Is there an way to make the JavaRDD<Object> from existing java >>>>> collection type List<Object>? >>>>> I know this can be done using scala , but i am looking how to do this >>>>> using java. >>>>> >>>>> >>>>> Regards >>>>> Jeetendra >>>>> >>>> >>>> >>> >>> >>> >> > > > >
