Session Window set max timeout

Is it possible to limit session windowing to a max of n seconds/hours etc? i.e. I would like a session window, but if a session runs for an unacceptable amount of time, I would like to close it. Thanks,

Flink Savepoint Config parameter

Hey, When running in EMR and taking a savepoint with flink cancel -s SAVEPOINT_DIR JOB_ID results in the following error Caused by: org.apache.flink.util.ConfigurationException: Config parameter 'Key: 'jobmanager.rpc.address' , default: null (deprecated keys: [])' is missing

Re: Flink on AWS EMR Protobuf

st, > Aljoscha > >> On 5. Sep 2017, at 23:52, ant burton <apburto...@gmail.com >> <mailto:apburto...@gmail.com>> wrote: >> >> Hello, >> >> Has anybody experienced the following error on AWS EMR 5.8.0 with Flink 1.3.1 >> >> java

Flink on AWS EMR Protobuf

Hello, Has anybody experienced the following error on AWS EMR 5.8.0 with Flink 1.3.1 java.lang.ClassCastException: org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetFileInfoRequestProto cannot be cast to com.google.protobuf.Message Thanks,

Re: Error submitting flink job

Is this of any help https://stackoverflow.com/questions/33890759/how-to-specify-overwrite-to-writeastext-in-apache-flink-streaming-0-10-0 fs.overwrite-files: true in your

Re: Flink Elastic Sink AWS ES

, > > Can you try this. > > curl -XGET 'http:///_cat/nodes?v=ip,port' > > This should give you ip and port > > On Mon, Aug 28, 2017 at 3:42 AM, ant burton <apburto...@gmail.com > <mailto:apburto...@gmail.com>> wrote: > Hi Arpit, > >

Re: Flink Elastic Sink AWS ES

Hi Arpit, The response fromm _nodes doesn’t contain an ip address in my case. Is this something that you experienced? > curl -XGET 'http:///_nodes' Thanks, > On 27 Aug 2017, at 14:32, ant burton <apburto...@gmail.com> wrote: > > Thanks! I'll check later this evening. &

Re: Flink Elastic Sink AWS ES

rs/elasticsearch.html > > <https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/elasticsearch.html> > > Please use Flink 1.3.1+ > > On Sat, Aug 26, 2017 at 3:00 PM, ant burton <apburto...@gmail.com > <mailto:apburto...@gmail.com>>

Flink Elastic Sink AWS ES

Hello, Has anybody been able to use the Flink Elasticsearch connector to sink data to AWS ES. I don’t believe this is possible as AWS ES only allows access to port 9200 (via port 80) on the master node of the ES cluster, and not port 9300 used by the the Flink Elasticsearch connector. The

Re: Access to datastream from BucketSink- RESOLVED

ng.connectors.fs.bucketing > > That one requires the implementation of 1 method with signature: > > Path getBucketPath(Clock clock, Path basePath, T element); > > which from what I understand from you requirements gives you access > to the element that you need. >

Re: Access to datastream from BucketSink

tPath(Clock clock, Path basePath, T element); > > which from what I understand from you requirements gives you access > to the element that you need. > > Cheers, > Kostas > >> On Aug 16, 2017, at 3:31 PM, ant burton <apburto...@gmail.com> wrote: >> >>

Re: Access to datastream from BucketSink

link for the BucketingSink, you can set your > own Bucketer using the setBucketer method. You do not have to > implement your own sink from scratch. > > Kostas > >> On Aug 16, 2017, at 1:39 PM, ant burton <apburto...@gmail.com >> <mailto:apburto...@gmail.com>&g

Re: Access to datastream from BucketSink

tml> > On 16 Aug 2017, at 12:24, Kostas Kloudas <k.klou...@data-artisans.com> wrote: > > Hi Ant, > > I think you can do it by implementing your own Bucketer. > > Cheers, > Kostas > > . >> On Aug 16, 2017, at 1:09 PM, ant burton <apbu

Re: Access to datastream from BucketSink

das <k.klou...@data-artisans.com> wrote: > > Hi Ant, > > I think you can do it by implementing your own Bucketer. > > Cheers, > Kostas > > . >> On Aug 16, 2017, at 1:09 PM, ant burton <apburto...@gmail.com> wrote: >> >> Hello, >> >&

Access to datastream from BucketSink

Hello, Given // Set StreamExecutionEnvironment final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); // Set checkpoints in ms env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime); // Add source (input

Re: Flink Data Streaming to S3

g 2017, at 07:32, vinay patil <vinay18.pa...@gmail.com> wrote: > > Hi, > > Yes, I am able to write to S3 using DataStream API. > > I have answered you the approach on SO > > Regards, > Vinay Patil > > On Mon, Aug 14, 2017 at 4:21 AM, ant burton [via Apache

Flink Data Streaming to S3

Hello, Has anybody been able to write to S3 when using the data streaming API ? I’m having this problem https://stackoverflow.com/questions/45655850/flink-s3-write-fails-unable-to-load-aws-credentials-from-any-provider-in-the-cha

No file system found with scheme s3

Hello, After following the instructions to set the S3 filesystem in the documentation (https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/aws.html#set-s3-filesystem ) I encountered the