[
https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880486#comment-16880486
]
Piotr Nowojski commented on FLINK-11082:
----------------------------------------
As [~NicoK] wrote, I think this shouldn't be an issue.
> Fix the calculation of backlog in PipelinedSubpartition
> -------------------------------------------------------
>
> Key: FLINK-11082
> URL: https://issues.apache.org/jira/browse/FLINK-11082
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Network
> Affects Versions: 1.5.6, 1.7.1
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The backlog of subpartition should indicate how many buffers are consumable,
> then the consumer could feedback the corresponding credits for transporting
> these buffers. But in current PipelinedSubpartitionimplementation, the
> backlog is increased by 1 when a BufferConsumer is added into
> PipelinedSubpartition, and decreased by 1 when a BufferConsumer is removed
> from PipelinedSubpartition. So the backlog only reflects how many buffers are
> retained in PipelinedSubpartition, which is not always equivalent to the
> number of consumable buffers.
> The backlog inconsistency might result in floating buffers misdistribution on
> consumer side, because the consumer would request floating buffers based on
> backlog value, then one floating buffer might not be used in
> RemoteInputChannel long time after requesting.
> Considering the solution, the last buffer in PipelinedSubpartition could only
> be consumable in the case of flush triggered or partition finished. So we
> could calculate the backlog precisely based on partition flushed/finished
> conditions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
