It's really Hadoop's support for S3. Hadoop FS semantics need directories, and S3 doesn't have a proper notion of directories.
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/s3native/NativeS3FileSystem.html You should leave them AFAIK. On Wed, Sep 24, 2014 at 6:41 PM, Shay Seng <s...@urbanengines.com> wrote: > Hi, > Why does RDD.saveAsObjectFile() to S3 leave a bunch of *_$folder$ empty > files around? Is it possible for the saveas to clean up? > > tks --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
