Re: Consuming Messages from Kafka

[Dominik Choma]

Hi,

You can check if any messages are going through dataflow on flink web dashboard
https://flink.apache.org/img/blog/new-dashboard-screenshot.png 
<https://flink.apache.org/img/blog/new-dashboard-screenshot.png>



Dominik Choma

> Wiadomość napisana przez Conlin, Joshua [USA] <conlin_jos...@bah.com> w dniu 
> 26 kwi 2016, o godz. 20:16:
> 
> re messages being sent to Kafka on that topic, I just never see anything in 
> Flink.  Any help/insight you could provide would be greatly appreciated.  If 
> it makes a difference this is running on YARN.  Also, here’s what I see in 
> the logs:

Re: Understanding Sliding Windows

[Dominik Choma]

Piyush,

You created sliding window witch is triggered every 10 seconds
Flink fires up this window every 10 seconds, without waiting at 5 min
buffer to be filled up
It seems to me that first argument is rather "maximum data buffer
retention" than " the initial threshold"

Dominik



Dominik

2016-04-26 12:16 GMT+02:00 Piyush Shrivastava :

> Hi all,
> I wanted to know how exactly sliding windows produce results in Flink.
> Suppose I create a sliding window of 5 minutes which is refreshed in every
> 10 seconds:
>
> .timeWindow(Time.minutes(5), Time.seconds(10))
>
> So in every 10 seconds we are looking at data from the past 5 minutes. But
> what happens before the initial 5 minutes have passed?
> Suppose we start the computation at 10:00. At 10:05 we will get the result
> for 10:00-10:05. But what are the results which we get in between this?
> i.e. at 10:00:10, 10:00:20 and so on.
> Basically why do Flink start producing results before the initial
> threshold has passed? What do these results signify?
>
> Thanks and Regards,
> Piyush Shrivastava 
> [image: WeboGraffiti]
> http://webograffiti.com
>