[
https://issues.apache.org/jira/browse/BEAM-8998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136942#comment-17136942
]
Beam JIRA Bot commented on BEAM-8998:
-------------------------------------
This issue was marked "stale-P2" and has not received a public comment in 14
days. It is now automatically moved to P3. If you are still affected by it, you
can comment and move it back to P2.
> Avoid excessive bundle progress polling in Dataflow Runner
> ----------------------------------------------------------
>
> Key: BEAM-8998
> URL: https://issues.apache.org/jira/browse/BEAM-8998
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Yichi Zhang
> Priority: P3
>
> Dataflow Java runner uses 0.1 secs interval for polling bundle progress from
> SDK Harness, and use the result to decide whether data transfer should be
> throttled. This can potentially overload SDK Harness.
> We should try to come up with a way to avoid the throttling and lower the
> bundle progress request frequency significantly.
>
> Code reference:
> frequency setting:
> [https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/fn/control/BeamFnMapTaskExecutor.java#L296]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
