The way to use Kryo serializer is similar as Scala, like below, the only
different is lack of convenient method "conf.registerKryoClasses", but it
should be easy to make one by yourself
conf=SparkConf()
conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
conf.set("spark.kryo.classesToRegister",
"com.example.YourClassA,com.example.YourClassB")
At 2016-08-23 02:00:41, "Eric Ho" <e...@analyticsmd.com> wrote:
I heard that Kryo will get phased out at some point but not sure which Spark
release.
I'm using PySpark, does anyone has any docs on how to call / use Kryo
Serializer in PySpark ?
Thanks.
--
-eric ho
