Thank you for your previous response Feng. It clears things up a little.
Regarding the inputQueueSize metric. What does it express after all?
Observing the inputQueueSize and inPoolUsage metrics in Prometheus, I
found that when the inPoolUsage is high, the inputQueueSize is low. This
observation does not match the description of the inputQueueSize.
Therefore, does this metric express the size of the input buffers of
each task, or does it express how full the input buffers of a task are?
Thank you in advance.
On 2023/11/28 16:11:59 Feng Jin wrote:
> Hi Dimitris
>
> Yes, the inputQueueLength metric can only represent the number of
buffers,
> and cannot obtain the actual size of the messages.
>
> Regarding the inputQueueSize metric, it ignores the LocalChannel.
> Therefore, when tasks are connected through the Forward partition
strategy,
> it will consistently display a value of 0.
> You may consider altering the partition strategy to the rebalance
strategy
> for testing purposes.st it.
>
>
> Best,
> Feng.
>
>
> On Tue, Nov 28, 2023 at 2:32 AM Dimitris Banelas via user <
> user@flink.apache.org> wrote:
>
> > As per the docs, the `inputQueueLength` metric refers to the number of
> > queued input buffers, and cannot be used on its own in order to
> > determine buffered records.
> >
> > For instance, if I know that there are 5 queued input buffers, I cannot
> > conclude anything regarding buffered records if the size of each queue
> > is not known.
> >
> >
> > On 2023/11/27 17:28:08 Feng Jin wrote:
> > > Hi Dimitris
> > >
> > > Maybe you can use the `inputQueueLength` metric.
> > >
> > >
> > > Best,
> > > Feng
> > >
> > > On Tue, Nov 28, 2023 at 12:07 AM Dimitris Mpanelas via user <
> > > user@flink.apache.org> wrote:
> > >
> > > > Hello,
> > > >
> > > > I am trying to determine the buffered records in the input
buffers of
> > a
> > > > task. I found the inputQueueSize metric. According to the docs it
> > is "The
> > > > real size of queued input buffers in bytes". The docs also state
> > that "The
> > > > size for local input channels is always 0 since the local
channel take
> > > > records directly from the output queue".
> > > >
> > > > What does this metric truly represent? In which cases should it be
> > 0 and
> > > > non 0? Is there a way to determine buffered records (or at least
> > something
> > > > similar) by using this metric?
> > > >
> > > > Dimitris.
> > > >
> > > >
> > >
> >
>
