RE: Why I didn't see the benefits of using KryoSerializer

that in Spark SQL? Thanks Yong From: iras...@cloudera.com Date: Fri, 20 Mar 2015 11:54:38 -0500 Subject: Re: Why I didn't see the benefits of using KryoSerializer To: java8...@hotmail.com CC: user@spark.apache.org Hi Yong, yes I think your analysis is correct. I'd imagine almost all serializers out

Re: Why I didn't see the benefits of using KryoSerializer

Hi Yong, yes I think your analysis is correct. I'd imagine almost all serializers out there will just convert a string to its utf-8 representation. You might be interested in adding compression on top of a serializer, which would probably bring the string size down in almost all cases, but then

RE: Why I didn't see the benefits of using KryoSerializer

I read the Spark code a little bit, trying to understand my own question. It looks like the different is really between org.apache.spark.serializer.JavaSerializer and org.apache.spark.serializer.KryoSerializer, both having the method named writeObject. In my test case, for each line of my text