There are many solutions to a problem. Also understand that sometimes your situation might be such. For ex what if you are accessing S3 from your Spark job running in your continuous integration server sitting in your data center or may be a box under your desk. And sometimes you are just trying something.
Also understand that sometimes you want answers to solve your problem at hand without redirecting you to something else. Understand what you suggested is an appropriate way of doing it, which I myself have proposed before, but that doesn't solve the OP's problem at hand. Regards Sab On 15-Mar-2016 8:27 pm, "Gourav Sengupta" <gourav.sengu...@gmail.com> wrote: > Oh!!! What the hell!!!! > > Please never use the URI > > *s3n://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY.*That is a major cause of > pain, security issues, code maintenance issues and ofcourse something that > Amazon strongly suggests that we do not use. Please use roles and you will > not have to worry about security. > > Regards, > Gourav Sengupta > > On Tue, Mar 15, 2016 at 2:38 PM, Sabarish Sasidharan < > sabarish....@gmail.com> wrote: > >> You have a slash before the bucket name. It should be @<bucket name>. >> >> Regards >> Sab >> On 15-Mar-2016 4:03 pm, "Yasemin Kaya" <godo...@gmail.com> wrote: >> >>> Hi, >>> >>> I am using Spark 1.6.0 standalone and I want to read a txt file from S3 >>> bucket named yasemindeneme and my file name is deneme.txt. But I am getting >>> this error. Here is the simple code >>> <https://gist.github.com/anonymous/6d174f8587f0f3fd2334> >>> Exception in thread "main" java.lang.IllegalArgumentException: Invalid >>> hostname in URI s3n://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY@ >>> /yasemindeneme/deneme.txt >>> at >>> org.apache.hadoop.fs.s3.S3Credentials.initialize(S3Credentials.java:45) >>> at >>> org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.initialize(Jets3tNativeFileSystemStore.java:55) >>> >>> >>> I try 2 options >>> *sc.hadoopConfiguration() *and >>> *sc.textFile("s3n://AWS_ACCESS_KEY_ID:AWS_SECRET_ACCESS_KEY@/yasemindeneme/deneme.txt/");* >>> >>> Also I did export AWS_ACCESS_KEY_ID= ..... >>> export AWS_SECRET_ACCESS_KEY= >>> But there is no change about error. >>> >>> Could you please help me about this issue? >>> >>> >>> -- >>> hiç ender hiç >>> >> >
