Re: Starting Flink cluster and running a job

2019-02-20 Thread Konstantin Knauf
Hi Boris, can you the relevant parts (dependencies) of your pom.xml? Did you also try without fixing the Kafka version, i.e. running with the Kafka client version provided by the Kafka connector of Flink? Gordon (cc) dealt with FLINK-8741. @Gordon: have you seen this issue with 1.6/1.7 before? C

Re: How to debug difference between Kinesis and Kafka

2019-02-20 Thread Congxian Qiu
Hi Stephen If the window has not been triggered ever, maybe you could investigate the watermark, maybe the doc[1][2] can be helpful. [1]  https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/stream/operators/windows.html#interaction-of-watermarks-and-windows [2]  https://ci.apache.org

Re: Starting Flink cluster and running a job

2019-02-20 Thread Boris Lublinsky
I found some more details on this The same error for the same application was reported about a year ago http://mail-archives.apache.org/mod_mbox/flink-user/201802.mbox/%3CCAE7GCT4pF74LwyY=tivzhquq50tkjjawfhaw+5phcsx+vos...@mail.gmail.com%3E

Re: How to use my custom log4j.properties when running minicluster in idea

2019-02-20 Thread Yun Tang
Hi Peibin I believe you could refer to the directory structure of flink/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/wordcount/WordCount.java, and the its log4j.properties file located at flink/flink-examples/flink-examples-streaming/src/main/resourc

Re: Jira issue Flink-11127

2019-02-20 Thread Boris Lublinsky
Also, The suggested workaround does not quite work. 2019-02-20 15:27:43,928 WARN akka.remote.ReliableDeliverySupervisor - Association with remote system [akka.tcp://flink-metrics@flink-taskmanager-1:6170] has failed, address is now gated for [50] ms. Reason: [Association

Re: Test FileUtilsTest.testDeleteDirectory failed when building Flink

2019-02-20 Thread Chesnay Schepler
I ran into a similar issue when i looked at other CI solutions; imo we shouldn't rely on the result of setWritable but instead actually verify whether the forbidden operation (i.e. creating/writing files) throws an error. Back then also created a JIRA: https://issues.apache.org/jira/browse/FL

Re: Assigning timestamps and watermarks several times, several datastreams?

2019-02-20 Thread Fabian Hueske
Hi, Watermarks of streams are independent as long as the streams are not connected with each other. When you union, join, or connect two streams in any other way, their watermarks are fused, which means that they are synced to the "slower" stream, i.e., the stream with the earlier watermarks. Bes