If you got “cannot Serialized” Exception, then you need to PixelGenerator as a Static class.
> On Nov 16, 2015, at 1:10 PM, Zhang, Jingyu <jingyu.zh...@news.com.au> wrote: > > Thanks, that worked for local environment but not in the Spark Cluster. > > > On 16 November 2015 at 16:05, Fengdong Yu <fengdo...@everstring.com > <mailto:fengdo...@everstring.com>> wrote: > Can you try : new PixelGenerator(startTime, endTime) ? > > > >> On Nov 16, 2015, at 12:47 PM, Zhang, Jingyu <jingyu.zh...@news.com.au >> <mailto:jingyu.zh...@news.com.au>> wrote: >> >> I want to pass two parameters into new java class from rdd.mapPartitions(), >> the code like following. >> ---Source Code---- >> >> Main method: >> >> /*the parameters that I want to pass into the PixelGenerator.class for >> selecting any items between the startTime and the endTime. >> >> */ >> >> int startTime, endTime; >> >> JavaRDD<PixelObject> pixelsObj = pixelsStr.mapPartitions(new >> PixelGenerator()); >> >> >> PixelGenerator.java >> >> public class PixelGenerator implements FlatMapFunction<Iterator<String>, >> PixelObject> { >> >> >> public Iterable<PixelObject> call(Iterator<String> lines) { >> >> ....... >> >> } >> >> Can anyone told me how to pass the startTime, endTime into PixelGenerator >> class? >> >> Many Thanks >> >> >> This message and its attachments may contain legally privileged or >> confidential information. It is intended solely for the named addressee. If >> you are not the addressee indicated in this message or responsible for >> delivery of the message to the addressee, you may not copy or deliver this >> message or its attachments to anyone. Rather, you should permanently delete >> this message and its attachments and kindly notify the sender by reply >> e-mail. Any content of this message and its attachments which does not >> relate to the official business of the sending company must be taken not to >> have been sent or endorsed by that company or any of its related entities. >> No warranty is made that the e-mail or attachments are free from computer >> virus or other defect. > > > > This message and its attachments may contain legally privileged or > confidential information. It is intended solely for the named addressee. If > you are not the addressee indicated in this message or responsible for > delivery of the message to the addressee, you may not copy or deliver this > message or its attachments to anyone. Rather, you should permanently delete > this message and its attachments and kindly notify the sender by reply > e-mail. Any content of this message and its attachments which does not relate > to the official business of the sending company must be taken not to have > been sent or endorsed by that company or any of its related entities. No > warranty is made that the e-mail or attachments are free from computer virus > or other defect.
