How does the *P1 *pipeline look like? Are there 2 downstream operators
reading from *Sin* (in this case the number of bytes would be measured
twice)?
On 14/06/2019 12:09, Andrea Spina wrote:
Sorry, I totally missed the version: flink-1.6.4, Streaming API
Il giorno ven 14 giu 2019 alle ore 11:08 Chesnay Schepler
<ches...@apache.org <mailto:ches...@apache.org>> ha scritto:
Which version of Flink are you using? There were some issues at
some point about double-counting.
On 14/06/2019 09:49, Andrea Spina wrote:
Dear Community, I'd like to ask for some details about bytes
related metrics in Flink. Precisely, I'm looking at *bytes sent*
and *bytes received *metrics: what I am recording is the following:
I am reading from a Kafka topic *A* records with schema *K* using
a source *Sin *belonging to a pipeline *P1*. The topic *A* is
filled with *K* data using a separated pipeline *p2* and a sink
*Sout*.
Basically what I register is that the /bytes sent/ metric of
*Sin* - available through the Flink UI -measure twice or more
respect of the /bytes received/ of *Sout. *For instance, if my
sink Sout records 10GB of /bytes received/ inbound, then my
source Sin emits between 20-25GB /bytes sent/.
Screenshot 2019-06-14 at 09.40.08.png
Is someone able to detail how these two metrics are calculated?
Thank you,
--
*Andrea Spina*
Software Engineer @ Radicalbit Srl
Via Borsieri 41, 20159, Milano - IT
--
*Andrea Spina*
Head of R&D @ Radicalbit Srl
Via Giovanni Battista Pirelli 11, 20124, Milano - IT
