Hi, I agree with you Richard. The point is that, looks like some classes which are used internally by Spark are not registered (for instance, the one I mentioned in the previous email is something I am not directly using). For those classes the serialization performance will be poor in according to how Spark works. How can I register all those classes?
cheers, From: Richard Startin [mailto:richardstar...@outlook.com] Sent: 10 January 2017 11:18 To: Enrico DUrso; user@spark.apache.org Subject: Re: Kryo On Spark 1.6.0 Hi Enrico, Only set spark.kryo.registrationRequired if you want to forbid any classes you have not explicitly registered - see http://spark.apache.org/docs/latest/configuration.html. Configuration - Spark 2.0.2 Documentation<http://spark.apache.org/docs/latest/configuration.html> spark.apache.org Spark Configuration. Spark Properties. Dynamically Loading Spark Properties; Viewing Spark Properties; Available Properties. Application Properties; Runtime Environment To enable kryo, you just need spark.serializer=org.apache.spark.serializer.KryoSerializer. There is some info here - http://spark.apache.org/docs/latest/tuning.html Cheers, Richard https://richardstartin.com/ ________________________________ From: Enrico DUrso <enrico.du...@everis.com<mailto:enrico.du...@everis.com>> Sent: 10 January 2017 11:10 To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Kryo On Spark 1.6.0 Hi, I am trying to use Kryo on Spark 1.6.0. I am able to register my own classes and it works, but when I set "spark.kryo.registrationRequired " to true, I get an error about a scala class: "Class is not registered: scala.collection.mutable.WrappedArray$ofRef". Any of you has already solved this issue in Java? I found the code to solve it in Scala, but unable to register this class in Java. Cheers, enrico ________________________________ CONFIDENTIALITY WARNING. This message and the information contained in or attached to it are private and confidential and intended exclusively for the addressee. everis informs to whom it may receive it in error that it contains privileged information and its use, copy, reproduction or distribution is prohibited. If you are not an intended recipient of this E-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute any portion of this E-mail. ________________________________ CONFIDENTIALITY WARNING. This message and the information contained in or attached to it are private and confidential and intended exclusively for the addressee. everis informs to whom it may receive it in error that it contains privileged information and its use, copy, reproduction or distribution is prohibited. If you are not an intended recipient of this E-mail, please notify the sender, delete it and do not read, act upon, print, disclose, copy, retain or redistribute any portion of this E-mail.