Davies Liu created SPARK-5091: --------------------------------- Summary: Hooks for PySpark tasks Key: SPARK-5091 URL: https://issues.apache.org/jira/browse/SPARK-5091 Project: Spark Issue Type: New Feature Components: PySpark Reporter: Davies Liu
Currently, it's not convenient to add package on executor to PYTHONPATH (we did not assume the environment of driver an executor are identical). It will be nice to have a hook to called before/after every tasks, then user could manipulate sys.path by pre-task-hooks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org