Re: [DISCUSS] FLIP-6 - Flink Deployment and Process Model - Standalone, Yarn, Mesos, Kubernetes, etc.

The design looks great - it solves for very diverse deployment modes, allows for heterogeneous TMs, and promotes job isolation. Some feedback: *Dispatcher* The dispatcher concept here expands nicely on what was introduced in the Mesos design doc (MESOS-1984). The most significant difference

Re: Discard out-of-order events

@Gyula: This is documented in the JavaDoc of the `allowedLateness(..)` method and in the docs: https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/windows.html#dealing-with-late-data @Kevin: Thanks for the explanation, I'll get back to you soon (sort of in a rush). Cheers, Max

Re: [VOTE] Release Apache Flink 1.1.0 (RC1)

When running "mvn clean verify" with Hadoop version 2.6.1 the Zookeeper/Leader Election tests fail with this: java.lang.NoSuchMethodError: org.apache.curator.utils.PathUtils.validatePath(Ljava/lang/String;)Ljava/lang/String; at

Re: FlinkKafkaConsumer09

Thanks! On Fri, Jul 29, 2016 at 11:43 AM, Gordon Tai (戴資力) wrote: > Hi Max, > > Sure, I was planning to do so, but wanted to see if it was a reasonable > feature to add before opening a JIRA :) > Here's the new JIRA: https://issues.apache.org/jira/browse/FLINK-4280 > >

[jira] [Created] (FLINK-4284) DataSet/CEP link to non-existant "Linking with Flink" section

Chesnay Schepler created FLINK-4284: --- Summary: DataSet/CEP link to non-existant "Linking with Flink" section Key: FLINK-4284 URL: https://issues.apache.org/jira/browse/FLINK-4284 Project: Flink

[jira] [Created] (FLINK-4283) ExecutionGraphRestartTest fails

Chesnay Schepler created FLINK-4283: --- Summary: ExecutionGraphRestartTest fails Key: FLINK-4283 URL: https://issues.apache.org/jira/browse/FLINK-4283 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4282) Add Offset Parameter to WindowAssigners

Aljoscha Krettek created FLINK-4282: --- Summary: Add Offset Parameter to WindowAssigners Key: FLINK-4282 URL: https://issues.apache.org/jira/browse/FLINK-4282 Project: Flink Issue Type:

Re: [DISCUSS][FLIP-4] Enhance Window Evictor in Flink

About processing time and timestamps: The timestamp is either set in the source of in an in-between TimestampAssigner that can be used with DataStream.assignTimestampsAndWatermarks(). However, the timestamp in the element is normally not a "processing-time timestamp". I think it might make sense

[jira] [Created] (FLINK-4281) Wrap all Calcite Exceptions in Flink Exceptions

Timo Walther created FLINK-4281: --- Summary: Wrap all Calcite Exceptions in Flink Exceptions Key: FLINK-4281 URL: https://issues.apache.org/jira/browse/FLINK-4281 Project: Flink Issue Type: Bug

Re: FlinkKafkaConsumer09

Hi Max, Sure, I was planning to do so, but wanted to see if it was a reasonable feature to add before opening a JIRA :) Here's the new JIRA: https://issues.apache.org/jira/browse/FLINK-4280 Regards, Gordon On Fri, Jul 29, 2016 at 4:03 PM, Maximilian Michels wrote: > Hi Tai, >

[jira] [Created] (FLINK-4279) [py] Set flink dependencies to provided

Chesnay Schepler created FLINK-4279: --- Summary: [py] Set flink dependencies to provided Key: FLINK-4279 URL: https://issues.apache.org/jira/browse/FLINK-4279 Project: Flink Issue Type:

Re: Discard out-of-order events

Goodmorning :-), Thank you for your answer. Let me explain my problem more thoroughly (maybe other options are possible here, not necessary with allowedLateness). The most compact description of my problem would be Stream Enrichment. More concrete, suppose I have two streams, where I want to

Re: Discard out-of-order events

Hi Max, So if I understand correctly the window operators now, by default, discard late elements? Is this documented somewhere? Gyula Maximilian Michels ezt írta (időpont: 2016. júl. 29., P, 10:40): > Hi! > > I'm not sure whether I understand your question. The purpose of

Re: Discard out-of-order events

Hi! I'm not sure whether I understand your question. The purpose of Event Time is to be able to process out-of-order events. Do you want to discard late elements? In the upcoming Flink 1.1.0 you can set the `allowedLateness` on a windowed stream. The default is 0, so late elements are discarded;

Re: FlinkKafkaConsumer09

Hi Tai, Should definitely be possible. Would you mind opening a JIRA issue with the description you posted? Thanks, Max On Thu, Jul 28, 2016 at 11:16 AM, Tai Gordon wrote: > Hi Kevin, > > Just a re-clarification: for Kafka 0.9 it would be “earliest”, & “smallest” > for the