Re: fs.s3a.endpoint not working

| Golden, CO 80401 > 303-275-4297 | caleb.phill...@nrel.gov > > From: Billy Watson <williamrwat...@gmail.com williamrwat...@gmail.com>> > Date: Tuesday, January 19, 2016 at 8:41 AM > To: Alexander Pivovarov <apivova...@gmail.com<mailto:apivova...@gmail.com &g

Re: fs.s3a.endpoint not working

>> Cc: Caleb Phillips <caleb.phill...@nrel.gov<mailto:caleb.phill...@nrel.gov>>, "user@hadoop.apache.org<mailto:user@hadoop.apache.org>" <user@hadoop.apache.org<mailto:user@hadoop.apache.org>> Subject: Re: fs.s3a.endpoint not working Stupid question, I a

Re: fs.s3a.endpoint not working

Stupid question, I assume you're using a URL that starts with s3a and that your custom endpoint supports s3a? William Watson Lead Software Engineer On Thu, Jan 14, 2016 at 1:57 PM, Alexander Pivovarov wrote: > http://www.jets3t.org/toolkit/configuration.html > On Jan 14,

Re: fs.s3a.endpoint not working

Hi Jonathan, >Im not totally following this thread from the beginning but I might be >able to help as I have some experience with Amazon EMR (elastic map >reduce) when working with custom jar files and s3 >Are you using EMR or something internal and offloading strage to s3? We have an

Re: fs.s3a.endpoint not working

Im not totally following this thread from the beginning but I might be able to help as I have some experience with Amazon EMR (elastic map reduce) when working with custom jar files and s3 Are you using EMR or something internal and offloading strage to s3? --- Regards, Jonathan Aquilina

Re: fs.s3a.endpoint not working

http://www.jets3t.org/toolkit/configuration.html On Jan 14, 2016 10:56 AM, "Alexander Pivovarov" wrote: > Add jets3t.properties file with s3service.s3-endpoint= to > /etc/hadoop/conf folder > > The folder with the file should be in HADOOP_CLASSPATH > > JetS3t library which

Re: fs.s3a.endpoint not working

Add jets3t.properties file with s3service.s3-endpoint= to /etc/hadoop/conf folder The folder with the file should be in HADOOP_CLASSPATH JetS3t library which is used by hadoop is looking for this file. On Dec 22, 2015 12:39 PM, "Phillips, Caleb" wrote: > Hi All, > >

Re: fs.s3a.endpoint not working

Hi Billy (and others), One of the threads suggested using the core-site.xml. Did you try putting your configuration in there? Yes, I did try that. I’ve also tried setting it dynamically in e.g., spark. I can verify that it is getting the configuration correctly: hadoop

Re: fs.s3a.endpoint not working

Hi All, Just wanted to send this out again since there was no response (admittedly, originally sent in the midst of the US holiday season) and it seems to be an issue that continues to come up (see e.g., the email from Han Ju on Jan 5). If anyone has successfully connected Hadoop to a non-AWS

Re: fs.s3a.endpoint not working

One of the threads suggested using the core-site.xml. Did you try putting your configuration in there? One thing I've noticed is that the AWS stuff is handled by an underlying library (I think jets3t in < 2.6 versions, forget what in 2.6+) and when I was trying to mess with stuff and spelunking

fs.s3a.endpoint not working

Hi All, New to this list. Looking for a bit of help: I'm having trouble connecting Hadoop to a S3-compatable (non AWS) object store. This issue was discussed, but left unresolved, in this thread: