Re: Maintaining watermarks per key, instead of per operator instance

Hello Fabian, Thank you very much. What is your opinion on the following solution: - Window data per time window, e.g. 15 minutes - using processing time as trigger, e.g. 15 minutes - which results in an aggregate over sensor values - then use cassandra to select the previous aggregate (as there

Many operations cause StackOverflowError with AWS EMR YARN cluster

Hello all, I have a pretty complicated plan file using the Flink Python API running on a AWS EMR cluster of m3.xlarge instances using YARN. The plan is for a dictionary learning algorithm and has to run a sequence of operations many times; each sequence involves bulk iterations with join operation

get trigger context from WindowFunction

Hi Flink Community, I have a question about knowing what triggered a window when a window function is executed. We have a case were multiple triggers can fire on a window and it would be helpful to understand which trigger is was once the window function is called. I'm not sure if there's a way

Global Sort + ZipWithIndex

Hello, I'm trying to assign a unique (and deterministic) ID to a globally sorted DataSet. Given a DataSet of String, I'm computing the frequency of each label as follows: val env = ExecutionEnvironment.getExecutionEnvironment val data = env.fromCollection(List("a","b","c","a","a","d","a","a","

[no subject]

Hello, I'm trying to assign a unique (and deterministic) ID to a globally sorted DataSet. Given a DataSet of String, I can compute the frequency of each label as follows: val env = ExecutionEnvironment.getExecutionEnvironment val data = env.fromCollection(List("a","b","c","a","a","d","a","a","

Re: Compile for Java 1.8

Thanks. Sounds weird I imagine the Flink team have a road-map to migrate to Java 1.8 On "Build From source ": "Flink requires *at least Java 7* to build. We recommend using Java 8." 2016-11-13 17:48 GMT+01:00 Andr

Re: Compile for Java 1.8

Hi Alberto, it seems that this class is not compatible with Java 8 new type inference rules, more specifically the 1.8 compiler picks a different overloaded method than 1.7. In TupleSerializer.java:112 the call to TypeSerializer.copy method in Java 7 resolves to public abstract T copy(T from, T

Re: Apache Flink Introduction Guide

Dear Anchit, Thanks for the feedback, i am developing another article for streaming, will include all the mentioned topics and mail here for the feedback again. On Sat, Nov 12, 2016 at 10:57 AM, Anchit Jatana < development.anc...@gmail.com> wrote: > Hi Manish, > > Appreciate the way you presente