Marcelo Vanzin created SPARK-5808: ------------------------------------- Summary: Assembly generated by sbt does not contain pyspark Key: SPARK-5808 URL: https://issues.apache.org/jira/browse/SPARK-5808 Project: Spark Issue Type: Bug Components: Build Reporter: Marcelo Vanzin Priority: Minor
When you generate the assembly with sbt, the py4j and pyspark files are not added to it. This makes pyspark not work when you run that assembly with yarn, since SPARK_HOME is not propagated in Yarn and thus PythonUtils.scala does not add the needed pyspark paths to PYTHONPATH. This is minor since all released bits are created by maven, so this should only affect developers who build with sbt and try pyspark on yarn. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org