Re: Event time window questions

[Sendoh]

setAutoWatermarkInterval configures how often the watermark is produced.
so if watermark is not proceeding, if you set shorter interval, you would
see t1, t1, t1, t1, t1 more often.
But what you would like to see is t1, t2, t3, t4

If you want to see count 0 when there is no incoming events,0 sounds for me
it's your use case, you can check sliding window.

I think seeing watermark in UI is possible now, or you can use debug mode to
see it.

The watermark you use won't wait for all topics(partitions). It's possible
if you implement your own watermark.

Cheers,

Sendoh



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Event time window questions

[Navneeth Krishnan]

Thanks Sendoh. Is there a way to advance watermark even when there are no
incoming events. What exactly does setAutoWatermarkInterval do?

Also I don't see the watermark displayed in flink dashboard.

Will the watermark advance only when there is data from all consuming kafka
topic and partitions? I have 3 topics with 3 partitions in each topic.

Thanks.

Regards,
Navneeth

On Tue, Jan 23, 2018 at 9:32 AM, Sendoh  wrote:

> Hi,
>
> you can write your own trigger and window, and implement whatever logic
> there.
> There are some examples
> https://github.com/apache/flink/blob/1875cac03042dad4a4c47b0de8364f
> 02fbe457c6/flink-streaming-java/src/main/java/org/apache/
> flink/streaming/api/windowing/triggers/
>
> If you don't see any event, it means window is not triggered.
>
> It would mean Watermark is not increasing. The issue can be the timestamp
> is
> not extracted correctly.
> Or, if you miss the trigger if use the window function doesn't have it.
>
> Cheers,
>
> Sendoh
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/
>

Re: Event time window questions

[Sendoh]

Hi, 

you can write your own trigger and window, and implement whatever logic
there.
There are some examples
https://github.com/apache/flink/blob/1875cac03042dad4a4c47b0de8364f02fbe457c6/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers/

If you don't see any event, it means window is not triggered.

It would mean Watermark is not increasing. The issue can be the timestamp is
not extracted correctly.
Or, if you miss the trigger if use the window function doesn't have it.

Cheers,

Sendoh 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/