[ https://issues.apache.org/jira/browse/SPARK-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15907366#comment-15907366 ]
Apache Spark commented on SPARK-19925: -------------------------------------- User 'yanboliang' has created a pull request for this issue: https://github.com/apache/spark/pull/17274 > SparkR spark.getSparkFiles fails on executor > -------------------------------------------- > > Key: SPARK-19925 > URL: https://issues.apache.org/jira/browse/SPARK-19925 > Project: Spark > Issue Type: Bug > Components: SparkR > Affects Versions: 2.1.0 > Reporter: Yanbo Liang > Priority: Critical > Attachments: error-log > > > SparkR function {{spark.getSparkFiles}} fails when it was called on > executors. For examples, the following R code will fail. (See error logs in > attachment.) > {code} > spark.addFile("./README.md") > seq <- seq(from = 1, to = 10, length.out = 5) > train <- function(seq) { > path <- spark.getSparkFiles("README.md") > print(path) > } > spark.lapply(seq, train) > {code} > However, we can run successfully with Scala API: > {code} > import org.apache.spark.SparkFiles > sc.addFile("./README.md”) > sc.parallelize(Seq(0)).map{ _ => SparkFiles.get("README.md")}.first() > {code} > and also successfully with Python API: > {code} > from pyspark import SparkFiles > sc.addFile("./README.md") > sc.parallelize(range(1)).map(lambda x: SparkFiles.get("README.md")).first() > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org