Sam Steingold created SPARK-7124:
------------------------------------
Summary: Add functions to check for file and directory existence
Key: SPARK-7124
URL: https://issues.apache.org/jira/browse/SPARK-7124
Project: Spark
Issue Type: Improvement
Components: Input/Output
Reporter: Sam Steingold
How do I check that a file or directory exists?
For file, I was told to do `sc.textFile().first()` which seems wrong:
# it initiates unnecessary i/o which could be huge (what is the file is binary
and has no newlines?)
# it fails for 0-length files (e.g., we write 0-length {{_SUCCESS}} files in
directories after they have been successfully written)
it appears that Spark needs bona file {{isFile}} and {{isDirectory}} methods.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
