Re: newbie unable to write to S3 403 forbidden error

at 7:05 PM To: Andrew Davidson Cc: "user @spark" Subject: Re: newbie unable to write to S3 403 forbidden error > > Make sure you are using s3 bucket in same region. Also I would access my > bucket this way s3n://bucketname/foldername. > > You can test privileges usi

Re: newbie unable to write to S3 403 forbidden error

Make sure you are using s3 bucket in same region. Also I would access my bucket this way s3n://bucketname/foldername. You can test privileges using the s3 cmd line client. Also, if you are using instance profiles you don't need to specify access and secret keys. No harm in specifying though. Reg

Re: newbie unable to write to S3 403 forbidden error

: "user @spark" Subject: Re: newbie unable to write to S3 403 forbidden error Hi Igor So I assume you are able to use s3 from spark? Do you use rdd.saveAsTextFile() ? How did you create your cluster? I.E. Did you use the spark-1.6.0/spark-ec2 script, EMR, or something else? I tried s

Re: newbie unable to write to S3 403 forbidden error

Igor Berman Date: Friday, February 12, 2016 at 4:53 AM To: Andrew Davidson Cc: "user @spark" Subject: Re: newbie unable to write to S3 403 forbidden error > String dirPath = "s3n://s3-us-west-1.amazonaws.com/com.pws.twitter/ > <http://s3-us-west-1.amazonaws.com/com

Re: newbie unable to write to S3 403 forbidden error

String dirPath = "s3n://s3-us-west-1.amazonaws.com/com.pws.twitter/*json” * not sure, but can you try to remove s3-us-west-1.amazonaws.com from path ? On 11 February 2016 at 23:15, Andy Davidson wrote: > I am using spark 1.6.0 in a cluster c

newbie unable to write to S3 403 forbidden error

I am using spark 1.6.0 in a cluster created using the spark-ec2 script. I am using the standalone cluster manager My java streaming app is not able to write to s3. It appears to be some for of permission problem. Any idea what the problem might be? I tried use the IAM simulator to test the polic