Did you try with s3a? It seems its more like an issue with hadoop.
Thanks
Best Regards
On Tue, Jul 21, 2015 at 2:31 PM, Schmirr Wurst <schmirrwu...@gmail.com>
wrote:
> It seems to work for the credentials , but the endpoint is ignored.. :
> I've changed it to sc.hadoopConfiguration.set("fs.s3n.endpoint","test.com
> ")
>
> And I continue to get my data from amazon, how could it be ? (I also
> use s3n in my text url)
>
> 2015-07-21 9:30 GMT+02:00 Akhil Das <ak...@sigmoidanalytics.com>:
> > You can add the jar in the classpath, and you can set the property like:
> >
> > sc.hadoopConfiguration.set("fs.s3a.endpoint","storage.sigmoid.com")
> >
> >
> >
> > Thanks
> > Best Regards
> >
> > On Mon, Jul 20, 2015 at 9:41 PM, Schmirr Wurst <schmirrwu...@gmail.com>
> > wrote:
> >>
> >> Thanks, that is what I was looking for...
> >>
> >> Any Idea where I have to store and reference the corresponding
> >> hadoop-aws-2.6.0.jar ?:
> >>
> >> java.io.IOException: No FileSystem for scheme: s3n
> >>
> >> 2015-07-20 8:33 GMT+02:00 Akhil Das <ak...@sigmoidanalytics.com>:
> >> > Not in the uri, but in the hadoop configuration you can specify it.
> >> >
> >> > <property>
> >> > <name>fs.s3a.endpoint</name>
> >> > <description>AWS S3 endpoint to connect to. An up-to-date list is
> >> > provided in the AWS Documentation: regions and endpoints. Without
> >> > this
> >> > property, the standard region (s3.amazonaws.com) is assumed.
> >> > </description>
> >> > </property>
> >> >
> >> >
> >> > Thanks
> >> > Best Regards
> >> >
> >> > On Sun, Jul 19, 2015 at 9:13 PM, Schmirr Wurst <
> schmirrwu...@gmail.com>
> >> > wrote:
> >> >>
> >> >> I want to use pithos, were do I can specify that endpoint, is it
> >> >> possible in the url ?
> >> >>
> >> >> 2015-07-19 17:22 GMT+02:00 Akhil Das <ak...@sigmoidanalytics.com>:
> >> >> > Could you name the Storage service that you are using? Most of them
> >> >> > provides
> >> >> > a S3 like RestAPI endpoint for you to hit.
> >> >> >
> >> >> > Thanks
> >> >> > Best Regards
> >> >> >
> >> >> > On Fri, Jul 17, 2015 at 2:06 PM, Schmirr Wurst
> >> >> > <schmirrwu...@gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I wonder how to use S3 compatible Storage in Spark ?
> >> >> >> If I'm using s3n:// url schema, the it will point to amazon, is
> >> >> >> there
> >> >> >> a way I can specify the host somewhere ?
> >> >> >>
> >> >> >>
> >> >> >>
> ---------------------------------------------------------------------
> >> >> >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> >> >> >> For additional commands, e-mail: user-h...@spark.apache.org
> >> >> >>
> >> >> >
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> >> >> For additional commands, e-mail: user-h...@spark.apache.org
> >> >>
> >> >
> >
> >
>
