Re: Apache Flink matrics are not alligned in the reporter

2021-08-23 Thread Jawad Tahir
Hi Chesnay,

Thank you for your message. My reporter interval is set to 1 second.

On Thu, 19 Aug 2021 at 19:51, Chesnay Schepler  wrote:

> What reporter interval do you have configured?
>
> On 19/08/2021 13:31, Jawad Tahir wrote:
> > Hi,
> >
> > I have defined Graphite as my matrics reporter with my Flink
> > (v1.13.2). My pipeline is pretty simple. It consists of one source,
> > one stateful operator (simple window aggregation), and one sink
> > (operations-playground, basically). I have set the parallel factor as
> > 2. The graph of the pipeline is as follows:
> >
> > [Flink pipeline][1]
> >
> > The program is running well and producing the correct results.
> > However, when I see the matrics, I see that source started sending the
> > records way after the system has started even though my sink was
> > producing correct results since the start of the job. Here is the
> > [graph][2] of uptime of the job and numRecordsOut of the source. As
> > far as I understood, Apache Flink sources' numRecordsOut should start
> > with uptime as my sink was producing correct results since the start.
> >
> >
> >   [1]: https://i.stack.imgur.com/rZm5h.png
> > <https://i.stack.imgur.com/rZm5h.png>
> >   [2]: https://i.stack.imgur.com/nlBoS.png
> > <https://i.stack.imgur.com/nlBoS.png>
>
>
>


Apache Flink matrics are not alligned in the reporter

2021-08-19 Thread Jawad Tahir
Hi,

I have defined Graphite as my matrics reporter with my Flink (v1.13.2). My
pipeline is pretty simple. It consists of one source, one stateful operator
(simple window aggregation), and one sink (operations-playground,
basically). I have set the parallel factor as 2. The graph of the pipeline
is as follows:

[Flink pipeline][1]

The program is running well and producing the correct results. However,
when I see the matrics, I see that source started sending the records way
after the system has started even though my sink was producing correct
results since the start of the job. Here is the [graph][2] of uptime of the
job and numRecordsOut of the source. As far as I understood, Apache Flink
sources' numRecordsOut should start with uptime as my sink was producing
correct results since the start.


  [1]: https://i.stack.imgur.com/rZm5h.png
  [2]: https://i.stack.imgur.com/nlBoS.png