Josh Rosen created SPARK-13926: ---------------------------------- Summary: Automatically use Kryo serializer when it is known to be safe Key: SPARK-13926 URL: https://issues.apache.org/jira/browse/SPARK-13926 Project: Spark Issue Type: Improvement Components: Shuffle Reporter: Josh Rosen Assignee: Josh Rosen
Because ClassTags are available when constructing ShuffledRDD we can use them to automatically use Kryo for shuffle serialization when the RDD's types are guaranteed to be compatible with Kryo (e.g. RDDs whose key, value, and/or combiner types are primitives, arrays of primitives, or strings). This is likely to result in a large performance gain for many RDD API workloads. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org