Does anyone know if I can save a RDD as a text file to a pre-created directory
in S3 bucket?
I have a directory created in S3 bucket: //nexgen-software/dev
When I tried to save a RDD as text file in this directory:
rdd.saveAsTextFile(s3n://nexgen-software/dev/output);
I got following
Your output folder specifies
rdd.saveAsTextFile(s3n://nexgen-software/dev/output);
So it will try to write to /dev/output which is as expected. If you create
the directory /dev/output upfront in your bucket, and try to save it to
that (empty) directory, what is the behaviour?
On Tue, Jan 27,
Subject: Re: SaveAsTextFile to S3 bucket
Your output folder specifies
rdd.saveAsTextFile(s3n://nexgen-software/dev/output);
So it will try to write to /dev/output which is as expected. If you create the
directory /dev/output upfront in your bucket, and try to save it to that
(empty) directory, what
By default, the files will be created under the path provided as the
argument for saveAsTextFile. This argument is considered as a folder in the
bucket and actual files are created in it with the naming convention
part-n, where n is the number of output partition.
On Mon, Jan 26, 2015 at
