[ https://issues.apache.org/jira/browse/SPARK-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964267#comment-14964267 ]
bin wang commented on SPARK-6527: --------------------------------- [~zhaozhang], this errors happens to me too while I am using Databricks' notebook. I have tons of images in a bucket, say `mybucket` wher when I do `binaryfiles('mybucket/*')`, it will error out with same message as yours. However, some of the images contain special characters that when I do `binaryfiles('mybucket/00*.jpg')` to restrict to a very small number of images, the command ran successfully. In that case, I think there is probably something picky about the file names containing certain characters. > sc.binaryFiles can not access files on s3 > ----------------------------------------- > > Key: SPARK-6527 > URL: https://issues.apache.org/jira/browse/SPARK-6527 > Project: Spark > Issue Type: Bug > Components: EC2, Input/Output > Affects Versions: 1.2.0, 1.3.0 > Environment: I am running Spark on EC2 > Reporter: Zhao Zhang > Priority: Minor > > The sc.binaryFIles() can not access the files stored on s3. It can correctly > list the number of files, but report "file does not exist" when processing > them. I also tried sc.textFile() which works fine. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org