Re: Event time didn't advance because of some idle slots

2018-07-31 Thread vino yang
Hi Soheil, Hequn has given you the usage of this method, see here : https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-kafka-base/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumerBase.java#L639 Thanks, vino. 2018-07-31 17:56 GMT+08:00 Soheil

Re: Event time didn't advance because of some idle slots

2018-07-31 Thread vino yang
Hi Soheil, The documentation of markAsTemporarilyIdle method is here : https://ci.apache.org/projects/flink/flink-docs-release-1.5/api/java/org/apache/flink/streaming/api/functions/source/SourceFunction.SourceContext.html#markAsTemporarilyIdle-- Thanks, vino. 2018-07-31 17:14 GMT+08:00 Hequn

Re: Event time didn't advance because of some idle slots

2018-07-31 Thread Hequn Cheng
Hi Soheil, You can set parallelism to 1 to solve the problem. Or use markAsTemporarilyIdle() as Fabian said(the link maybe is

Re: Event time didn't advance because of some idle slots

2018-07-31 Thread Fabian Hueske
Hi, If you are using a custom source, you can call SourceContext.markAsTemporarilyIdle() to indicate that a task is currently not producing new records [1]. Best, Fabian 2018-07-31 8:50 GMT+02:00 Reza Sameei : > It's not a real solution; but why you don't change the parallelism for > your

Re: Event time didn't advance because of some idle slots

2018-07-31 Thread Reza Sameei
It's not a real solution; but why you don't change the parallelism for your `SourceFunction`? On Tue, Jul 31, 2018 at 10:51 AM Soheil Pourbafrani wrote: > In Flink Event time mode, I use the periodic watermark to advance event > time. Every slot extract event time from the incoming message and

Event time didn't advance because of some idle slots

2018-07-31 Thread Soheil Pourbafrani
In Flink Event time mode, I use the periodic watermark to advance event time. Every slot extract event time from the incoming message and to emit watermark, subtract it a network delay, say 3000ms. public Watermark getCurrentWatermark() { return new Watermark(MAX_TIMESTAMP - DELEY);