holdenk created SPARK-15369: ------------------------------- Summary: Investigate selectively using Jython for parts of PySpark Key: SPARK-15369 URL: https://issues.apache.org/jira/browse/SPARK-15369 Project: Spark Issue Type: Improvement Components: PySpark Reporter: holdenk Priority: Minor
Transfering data from the JVM to the Python executor can be a substantial bottleneck. While JYthon is not suitable for all UDFs or map functions, it may be suitable for some simple ones. We should investigate the option of using JYthon to accelerate these small functions. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org