[
https://issues.apache.org/jira/browse/SPARK-10500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14973530#comment-14973530
]
Sun Rui commented on SPARK-10500:
---------------------------------
yes, I am working on this
> sparkr.zip cannot be created if $SPARK_HOME/R/lib is unwritable
> ---------------------------------------------------------------
>
> Key: SPARK-10500
> URL: https://issues.apache.org/jira/browse/SPARK-10500
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 1.5.0
> Reporter: Jonathan Kelly
>
> As of SPARK-6797, sparkr.zip is re-created each time spark-submit is run with
> an R application, which fails if Spark has been installed into a directory to
> which the current user doesn't have write permissions. (e.g., on EMR's
> emr-4.0.0 release, Spark is installed at /usr/lib/spark, which is only
> writable by root.)
> Would it be possible to skip creating sparkr.zip if it already exists? That
> would enable sparkr.zip to be pre-created by the root user and then reused
> each time spark-submit is run, which I believe is similar to how pyspark
> works.
> Another option would be to make the location configurable, as it's currently
> hardcoded to $SPARK_HOME/R/lib/sparkr.zip. Allowing it to be configured to
> something like the user's home directory or a random path in /tmp would get
> around the permissions issue.
> By the way, why does spark-submit even need to re-create sparkr.zip every
> time a new R application is launched? This seems unnecessary and inefficient,
> unless you are actively developing the SparkR libraries and expect the
> contents of sparkr.zip to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
