Nico Kruber created FLINK-10762:
-----------------------------------
Summary: Do not try to request partitions in SingleInputGate per
getRecord
Key: FLINK-10762
URL: https://issues.apache.org/jira/browse/FLINK-10762
Project: Flink
Issue Type: Improvement
Components: Network
Affects Versions: 1.7.0
Reporter: Nico Kruber
Fix For: 1.8.0
Every call to {{SingleInputGate#getNextBufferOrEvent()}} will also try to
request partitions via {{SingleInputGate#requestPartitions()}}. However, in the
current design, this must be under the {{requestLock}} which is costly and
unnecessary.
I propose a redesign to change that and make the
{{SingleInputGate#requestPartitions()}} a one-time only thing that should be
called *before* going into any loop asking for new records. In that case, we
could keep the lock and ignore and performance-related improvements.
Alternatively, if we had a separate {{unrequestedPartitions}} list aside to
{{inputChannels}} and only go through these when executing
{{SingleInputGate#requestPartitions()}}, we would change {{updateInputChannel}}
to always request the newly created partition and will not have the concurrency
issue between these two methods anymore. This {{unrequestedPartitions}} list
would basically also make {{requestedPartitionsFlag}} obsolete if we, for
example, set it to {{Optional.empty}} after finishing the requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
