Re: Regarding KryoSerialization in Spark

2015-04-30 Thread Sandy Ryza
Hi Twinkle,

Registering the class makes it so that writeClass only writes out a couple
bytes, instead of a full String of the class name.

-Sandy

On Thu, Apr 30, 2015 at 4:13 AM, twinkle sachdeva 
twinkle.sachd...@gmail.com wrote:

 Hi,

 As per the code, KryoSerialization used writeClassAndObject method, which
 internally calls writeClass method, which will write the class of the
 object while serilization.

 As per the documentation in tuning page of spark, it says that registering
 the class will avoid that.

 Am I missing something or there is some issue with the documentation???

 Thanks,
 Twinkle



Re: Regarding KryoSerialization in Spark

2015-04-30 Thread twinkle sachdeva
Thanks for the info.


On Fri, May 1, 2015 at 12:10 AM, Sandy Ryza sandy.r...@cloudera.com wrote:

 Hi Twinkle,

 Registering the class makes it so that writeClass only writes out a couple
 bytes, instead of a full String of the class name.

 -Sandy

 On Thu, Apr 30, 2015 at 4:13 AM, twinkle sachdeva 
 twinkle.sachd...@gmail.com wrote:

 Hi,

 As per the code, KryoSerialization used writeClassAndObject method, which
 internally calls writeClass method, which will write the class of the
 object while serilization.

 As per the documentation in tuning page of spark, it says that registering
 the class will avoid that.

 Am I missing something or there is some issue with the documentation???

 Thanks,
 Twinkle