Re: Save data to different S3

:jingyu.zh...@news.com.au>>, user <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: Save data to different S3 On 30 Oct 2015, at 18:05, William Li <a-...@expedia.com<mailto:a-...@expedia.com>> wrote: Thanks for your response. My secret has a back splas

Re: Save data to different S3

On 30 Oct 2015, at 18:05, William Li > wrote: Thanks for your response. My secret has a back splash (/) so it didn’t work… that's a recurrent problem with the hadoop/java s3 clients. Keep trying to regenerate a secret until you get one that works

Re: Save data to different S3

expedia.com>> Cc: user <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: Save data to different S3 Try s3://aws_key:aws_secret@bucketName/folderName with your access key and secret to save the data. On 30 October 2015 at 10:55, William Li <a-...@expedia.com<mailto

Save data to different S3

Hi - I have a simple app running fine with Spark, it reads data from S3 and performs calculation. When reading data from S3, I use hadoopConfiguration.set for both fs.s3n.awsAccessKeyId, and the fs.s3n.awsSecretAccessKey to it has permissions to load the data from customer sources. However,

Re: Save data to different S3

Try s3://aws_key:aws_secret@bucketName/folderName with your access key and secret to save the data. On 30 October 2015 at 10:55, William Li wrote: > Hi – I have a simple app running fine with Spark, it reads data from S3 > and performs calculation. > > When reading data from