Hi, Owen,
it is fixed after registering manually:
conf.registerKryoClasses(Array(Class.forName("[D")))I believe that Kyro (latest version) have supported double[] : addDefaultSerializer(double[].class, DoubleArraySerializer.class); <https://github.com/EsotericSoftware/kryo/blob/e2fe2a4c339f48120e71ae61967f76ddfcf4ba3a/src/com/esotericsoftware/kryo/Kryo.java#L197> Why does it break in spark? On Sun, Jan 8, 2017 at 6:03 PM, Sean Owen <so...@cloudera.com> wrote: > Double[] is not of the same class as double[]. Kryo should already know > how to serialize double[], but I doubt Double[] is registered. > > The error does seem to clearly indicate double[] though. That surprises > me. Can you try manually registering it to see if that fixes it? > But then I'm not sure why tests wouldn't catch this. > > On Sun, Jan 8, 2017 at 7:30 AM smartzjp <zjp_j...@163.com> wrote: > >> You can have a try the following code. >> >> ObjectArraySerializer serializer = new ObjectArraySerializer(kryo, Double >> [].class); >> kryo.register(Double[].class, serializer); >> >> >> ----------- >> >> Hi, all. >> I enable kyro in spark with spark-defaults.conf: >> spark.serializer org.apache.spark.serializer. >> KryoSerializer >> spark.kryo.registrationRequired true >> >> A KryoException is raised when a logistic regression algorithm is running: >> Note: To register this class use: kryo.register(double[].class); >> Serialization trace: >> currL1 (org.apache.spark.mllib.stat.MultivariateOnlineSummarizer) >> at com.esotericsoftware.kryo.serializers.FieldSerializer$ >> ObjectField.write(FieldSerializer.java:585) >> at com.esotericsoftware.kryo.serializers.FieldSerializer. >> write(FieldSerializer.java:213) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo. >> java:568) >> at com.twitter.chill.Tuple2Serializer.write( >> TupleSerializers.scala:36) >> at com.twitter.chill.Tuple2Serializer.write( >> TupleSerializers.scala:33) >> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo. >> java:568) >> >> My question is: >> Doesn't double[].class be supported by default? >> >> Thanks. >> >>
