[
https://issues.apache.org/jira/browse/APEXCORE-570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15630749#comment-15630749
]
Munagala V. Ramanath commented on APEXCORE-570:
-----------------------------------------------
I think part of the problem is that sometimes applications are written where
upstream operators maintain state that will stay a reasonable size as long as
data flows through the pipeline normally but if the downstream slows down,
state starts to get bigger and bigger to the point where the operator is deemed
hung and killed perhaps because the checkpointing is taking too long, or
perhaps because it blows memory limits because spilling to disk is not
happening fast enough. In such
cases, this sort of throttling mechanism can be a useful coping mechanism.
> Prevent upstream operators from getting too far ahead when downstream
> operators are slow
> ----------------------------------------------------------------------------------------
>
> Key: APEXCORE-570
> URL: https://issues.apache.org/jira/browse/APEXCORE-570
> Project: Apache Apex Core
> Issue Type: Improvement
> Reporter: Pramod Immaneni
> Assignee: Pramod Immaneni
>
> If the downstream operators are slower than upstream operators then the
> upstream operators will get ahead and the gap can continue to increase.
> Provide an option to slow down or temporarily pause the upstream operators
> when they get too far ahead.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
