Re: S3 Access in eu-central-1

2018-01-02 Thread Nico Kruber
Sorry for the late response, but I finally got around adding this workaround to our "common issues" section with PR https://github.com/apache/flink/pull/5231 Nico On 29/11/17 09:31, Ufuk Celebi wrote: > Hey Dominik, > > yes, we should definitely add this to the docs. > > @Nico: You recently upd

Re: S3 Access in eu-central-1

2017-11-29 Thread Ufuk Celebi
Hey Dominik, yes, we should definitely add this to the docs. @Nico: You recently updated the Flink S3 setup docs. Would you mind adding these hints for eu-central-1 from Steve? I think that would be super helpful! Best, Ufuk On Tue, Nov 28, 2017 at 10:00 PM, Dominik Bruhn wrote: > Hey Stephan

Re: S3 Access in eu-central-1

2017-11-28 Thread Dominik Bruhn
Hey Stephan, Hey Steve, that was the right hint, adding that open to the Java-Options fixed the problem. Maybe we should add this somehow to our Flink Wiki? Thanks! Dominik On 28/11/17 11:55, Stephan Ewen wrote: Got a pointer from Steve that this is answered on Stack Overflow here: https://s

Re: S3 Access in eu-central-1

2017-11-28 Thread Stephan Ewen
Got a pointer from Steve that this is answered on Stack Overflow here: https://stackoverflow.com/questions/36154484/aws-java- sdk-manually-set-signature-version Flink 1.4 contains a specially bundled "fs-s3-hadoop" with smaller no footprint, compatible across Hadoop versions, and based on a later

Re: S3 Access in eu-central-1

2017-11-27 Thread Stephan Ewen
Hi! The endpoint config entry looks correct. I was looking at this issue to see if there are pointers to anything else, but it looks like the explicit endpoint entry is the most important thing: https://issues.apache.org/jira/browse/HADOOP-13324 I cc-ed Steve Loughran, who is Hadoop's S3 expert (

Re: S3 Access in eu-central-1

2017-11-27 Thread Dominik Bruhn
Hey, can anyone give a hint? Does anyone have flink running with an S3 Bucket in Frankfurt/eu-central-1 and can share his config and setup? Thanks, Dominik > On 22. Nov 2017, at 17:52, domi...@dbruhn.de wrote: > > Hey everyone, > I'm trying since hours to get Flink 1.3.2 (downloaded for hadoop

Re: S3 Access in eu-central-1

2017-11-22 Thread Timo Walther
@Patrick: Do you have an advice? Am 11/22/17 um 5:52 PM schrieb domi...@dbruhn.de: Hey everyone, I'm trying since hours to get Flink 1.3.2 (downloaded for hadoop 2.7) to snapshot/checkpoint to an S3 bucket which is hosted in the eu-central-1 region. Everything works fine for other regions. I'

S3 Access in eu-central-1

2017-11-22 Thread dominik
Hey everyone, I'm trying since hours to get Flink 1.3.2 (downloaded for hadoop 2.7) to snapshot/checkpoint to an S3 bucket which is hosted in the eu-central-1 region. Everything works fine for other regions. I'm running my job on a JobTracker in local mode. I googled the internet and found seve