Hi, I have a job like 'Source -> assignmentTimestamp -> flatmap ->  Window
-> Sink' and I get back pressure from 'Source' to the 'FlatMap' operators
form the 'BackPressure' tab in the Web UI.
When trying to find which operator is the source of back pressure, I use
metrics provided by the Web UI, specifically, 'inPoolUsage' and
'outPoolUsage'.
Firstly, As far as I know, when both of the metrics are 0, the operator
should not be defined as 'back pressured', but when I check the
'AssignmentTimestamp' operator, where 8 subtasks running, I find 1 or 2 of
them have 0 value about the back pressure index, and the others have the
index higher than 0.80, and all of them are marked  in 'HIGH' status.
However, the two metrics, 'in/outPoolUsage', are always be 0. So maybe the
operator is not back pressured actually?  Or is there any problem with my
Flink WebUI?
Second question is, from my experience, I think the source of the back
pressure should be the Window operator because the outPoolUsage of the
'FlatMap' are 1, and the 'Window' is the first downstream operator from the
'Flatmap', but the inPoolUsage and the outPoolUsage are also 0. So the
cause of the back pressure should be the network bottleneck between window
and flatmap? Am I right?
Thanks for your reading, and I'm looking forward for your ideas.

Haocheng

Reply via email to