Re: Serialization Problem in Spark Program

quot;;; > *Send time:* Thursday, Mar 26, 2015 3:23 PM > *To:* "donhoff_h"<165612...@qq.com>; > *Cc:* "user"; > *Subject: * Re: Serialization Problem in Spark Program > > Try registering your MyObject[] with Kryo. > On 25 Mar 2015 13:17, "donhoff

Re: Serialization Problem in Spark Program

Try registering your MyObject[] with Kryo. On 25 Mar 2015 13:17, "donhoff_h" <165612...@qq.com> wrote: > Hi, experts > > I wrote a very simple spark program to test the KryoSerialization > function. The codes are as following: > > object TestKryoSerialization { > def main(args: Array[String]) {

Re: Serialization Problem in Spark Program

you also need to register *array*s of MyObject. so change: conf.registerKryoClasses(Array(classOf[MyObject])) to conf.registerKryoClasses(Array(classOf[MyObject], classOf[Array[MyObject]])) On Wed, Mar 25, 2015 at 2:44 AM, donhoff_h <165612...@qq.com> wrote: > Hi, experts > > I wrote a very

Serialization Problem in Spark Program

Hi, experts I wrote a very simple spark program to test the KryoSerialization function. The codes are as following: object TestKryoSerialization { def main(args: Array[String]) { val conf = new SparkConf() conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer") c

Re: Serialization problem in Spark

View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Serialization-problem-in-Spark-tp7049p8205.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Serialization problem in Spark

t the complaint is for "rg.apache..." (missing o is > not a typo). > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Serialization-problem-in-Spark-tp7049p8123.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >

Re: Serialization problem in Spark

strange in that the complaint is for "rg.apache..." (missing o is not a typo). -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Serialization-problem-in-Spark-tp7049p8123.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Serialization problem in Spark

park-user-list.1001560.n3.nabble.com/Serialization-problem-in-Spark-tp7049p7975.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Serialization problem in Spark

Where are you getting serialization error. Its likely to be a different problem. Which class is not getting serialized? Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi On Thu, Jun 5, 2014 at 6:32 PM, Vibhor Banga wrote: >

Re: Serialization problem in Spark

Any inputs on this will be helpful. Thanks, -Vibhor On Thu, Jun 5, 2014 at 3:41 PM, Vibhor Banga wrote: > Hi, > > I am trying to do something like following in Spark: > > JavaPairRDD eventRDD = hBaseRDD.map(new > PairFunction, byte[], MyObject >() { > @Override > public

Serialization problem in Spark

Hi, I am trying to do something like following in Spark: JavaPairRDD eventRDD = hBaseRDD.map(new PairFunction, byte[], MyObject >() { @Override public Tuple2 call(Tuple2 immutableBytesWritableResultTuple2) throws Exception { return new Tuple2(immutableBytes