[jira] [Commented] (SPARK-14859) [PYSPARK] Make Lambda Serializer Configurable
[ https://issues.apache.org/jira/browse/SPARK-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254665#comment-15254665 ] Nick White commented on SPARK-14859: I've got a PR for this here: https://github.com/apache/spark/pull/12620 > [PYSPARK] Make Lambda Serializer Configurable > - > > Key: SPARK-14859 > URL: https://issues.apache.org/jira/browse/SPARK-14859 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 2.0.0 >Reporter: Nick White > > Currently lambdas (e.g. used in RDD.map) are serialized by a hardcoded > reference to the CloudPickleSerializer. The serializer should be > configurable, as these lambdas may contain complex objects that need custom > serialization. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-14859) [PYSPARK] Make Lambda Serializer Configurable
[ https://issues.apache.org/jira/browse/SPARK-14859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15254666#comment-15254666 ] Apache Spark commented on SPARK-14859: -- User 'njwhite' has created a pull request for this issue: https://github.com/apache/spark/pull/12620 > [PYSPARK] Make Lambda Serializer Configurable > - > > Key: SPARK-14859 > URL: https://issues.apache.org/jira/browse/SPARK-14859 > Project: Spark > Issue Type: Improvement > Components: PySpark >Affects Versions: 2.0.0 >Reporter: Nick White > > Currently lambdas (e.g. used in RDD.map) are serialized by a hardcoded > reference to the CloudPickleSerializer. The serializer should be > configurable, as these lambdas may contain complex objects that need custom > serialization. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
