Re: A proposal for Spark 2.0

2015-11-11 Thread hitoshi
It looks like Chill is willing to upgrade their Kryo to 3.x if Spark and Hive will. As it is now Spark, Chill, and Hive have Kryo jar but it really can't be used because Kryo 2 can't serdes some classes. Since Spark 2.0 is a major release, it really would be nice if we can resolve the Kryo issue.

Re: A proposal for Spark 2.0

2015-11-11 Thread hitoshi
Resending my earlier message because it wasn't accepted. Would like to add a proposal to upgrade jars when they do not break APIs and fixes a bug. To be more specific, I would like to see Kryo to be upgraded from 2.21 to 3.x. Kryo 2.x has a bug (e.g.SPARK-7708) that is blocking it usage in