Deterministic Update

Is there any possibility to trigger sink operator on completion of checkpoint? -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Deterministic-Update-tp13580.html Sent from the Apache Flink User Mailing List archive. mailing list archive at

Re: Flink streaming Python

Vote for python +1. I find it can't support kafka source from code. -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-streaming-Python-tp13573p13578.html Sent from the Apache Flink User Mailing List archive. mailing list archive at

回复：Question regarding configuring number of network buffers

Hi Ray, For your question : Does that say that each parallel task inside the  TaskManager talk to all parallel tasks inside the same TaskManager or to all  parallel tasks across all task managers? Each task will talk to all parallel upstream and downstream tasks that both include the same

Re: How to run a Flink job in EMR?

Ah, maybe (1) wasn’t entirely clear so here’s the copy/pasted example with what I suggested: HadoopJarStepConfig copyJar = new HadoopJarStepConfig() .withJar("command-runner.jar") .withArgs("bash","-c", "aws s3 cp s3://mybucket/myjar.jar /home/hadoop" ); From: "Foster,

Re: How to run a Flink job in EMR?

1) Since the jar is only required on the master node you should be able to just run a step with a very simple script like ‘bash –c “aws s3 cp s3://mybucket/myjar.jar .”’ So if you were to do that using the step similar to outlined in the EMR documentation, but replacing withArgs with the

Flink streaming Python

Hi I have asked the same question back in Jan 2016 and checking again with community to see if there is any update or plan for supporting streaming Flink in python.

Re: Guava version conflict

Yes, those should not be in the flink-dist jar, so the root reason should be that the shading isn’t working properly for your custom build. If possible, could you try building Flink again with a lower Maven version as specified in the doc, and see if that works? If so, it could be that Maven

Re: Guava version conflict

What I did was to take the sources of the new ES connector and I took them into my code. Flink was compiled with maven 3.3+ but I did the double compilation as specified in the Flink build section. In flink dist I see guava classes, e.g.: com/google/common/util/concurrent/MoreExecutors$1.class

Re: Painful KryoException: java.lang.IndexOutOfBoundsException on Flink Batch Api scala

Hi Andrea, I did some quick issue searching, and it seems like this is a frequently asked issue on Kryo: https://github.com/EsotericSoftware/kryo/issues/428. I can’t be sure at the moment if the resolution / workaround mentioned in there makes sense, I’ll have to investigate a bit more. Also,

Re: Queries regarding FlinkCEP

Hi Dawid, Yes, now I understood what you meant. Although I added exactly the input you asked me to and I still get no alerts. I also observed that I am not getting alerts even with normal ordering of timestamp and with ascedingTimestampExtractor. I am adding an image where I entered the data

Re: Guava version conflict

Hi Flavio, Could there be another dependency in your job that requires a conflicting version (w.r.t. ES 2.4.1) of Guava? I’ve just double checked the flink-dist jar, there doesn’t seem to be any non-shaded Guava dependencies there, so the conflict should not have been caused by Flink. Cheers,

Guava version conflict

Hi to all, I'm trying to use the new ES connector to index data from Flink (with ES 2.4.1). When I try to run it from Eclipse everything is ok, when I run it from the cluster I get the following exception: java.lang.NoSuchMethodError: com.google.common.util.

Painful KryoException: java.lang.IndexOutOfBoundsException on Flink Batch Api scala

Good afternoon dear Community, Since few days I'm really struggling to understand the reason behind this KryoException. Here the stack trace. 2017-06-07 10:18:52,514 ERROR org.apache.flink.runtime.operators.BatchTask - Error in task code: CHAIN GroupReduce (GroupReduce at

Re: Queryable State Client with 1.3.0-rc0

Hi Claudio, Quick question: what exactly was your call for getting the local environment with web UI? Did you also have a custom Configuration where you specified, for example, that the queryable state server should be enabled? I can make an example work where I start a local cluster in one

Re: AsyncCollector Does not release the thread (1.2.1)

Hi Steve, I’m assuming you are using Flink 1.2.x? If yes, then I’m afraid you re-discovered this issue: https://issues.apache.org/jira/browse/FLINK-6435 . It was fixed in Flink 1.3.0. Is it possible for you to update to that version or do you

Re: Flink and swapping question

I forgot to mention that my jobs are all batch (at the moment). Do you think that this problem could be related to - http://www.evanjones.ca/java-bytebuffer-leak.html#comment-3240054880 - and http://www.evanjones.ca/java-native-leak-bug.html Kurt told me also to add "env.java.opts:

Re: Running job in "dry mode"?

On 07/06/2017 10:27, Maciek Próchniak wrote: On 07/06/2017 10:07, Tzu-Li (Gordon) Tai wrote: Hi Maciek, Is there any particular reason why you do not wish to start running the Kafka sources on the test run? Otherwise, it would be perfectly fine to start the test job for testing to see

[DISCUSS] Removal of twitter-inputformat

Hello, I'm proposing to remove the Twitter-InputFormat in FLINK-6710 , with an open PR you can find here . The PR currently has a +1 from Robert, but Timo raised some concerns saying that it is useful

Re: Methods that trigger execution

Hi, I’m afraid I don’t know that part well enough. What’s the percentage in slowdown? (7 seconds alone doesn’t say anything) Maybe Till (in cc) knows more since he used to work on the ML part. Best, Aljoscha > On 6. Jun 2017, at 17:45, Borja wrote: > > *Thank so