[
https://issues.apache.org/jira/browse/FLINK-10142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587618#comment-16587618
]
ASF GitHub Bot commented on FLINK-10142:
----------------------------------------
NicoK commented on a change in pull request #6555: [FLINK-10142][network]
reduce locking around credit notification
URL: https://github.com/apache/flink/pull/6555#discussion_r211658191
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/consumer/RemoteInputChannel.java
##########
@@ -509,33 +514,40 @@ void onSenderBacklog(int backlog) throws IOException {
}
public void onBuffer(Buffer buffer, int sequenceNumber, int backlog)
throws IOException {
- boolean success = false;
+ boolean recycleBuffer = true;
try {
+
+ final boolean wasEmpty;
synchronized (receivedBuffers) {
- if (!isReleased.get()) {
- if (expectedSequenceNumber ==
sequenceNumber) {
- int available =
receivedBuffers.size();
+ // Similar to notifyBufferAvailable(), make
sure that we never add a buffer
+ // after releaseAllResources() released all
buffers from receivedBuffers
+ // (see above for details).
+ if (isReleased.get()) {
+ return;
+ }
- receivedBuffers.add(buffer);
- expectedSequenceNumber++;
+ if (expectedSequenceNumber != sequenceNumber) {
+ onError(new
BufferReorderingException(expectedSequenceNumber, sequenceNumber));
+ return;
+ }
- if (available == 0) {
- notifyChannelNonEmpty();
- }
+ wasEmpty = receivedBuffers.isEmpty();
+ receivedBuffers.add(buffer);
+ recycleBuffer = false;
+ }
- success = true;
- } else {
- onError(new
BufferReorderingException(expectedSequenceNumber, sequenceNumber));
- }
- }
+ ++expectedSequenceNumber;
+
+ if (wasEmpty) {
+ notifyChannelNonEmpty();
Review comment:
actually, I didn't measure every single change's performance impact, but
having `notifyChannelNonEmpty()` under the lock would mean that the lock around
`RemoteInputChannel#receivedBuffers` would compete with the lock around
`SingleInputGate#inputChannelsWithData`
-> that change is one of the two lock contention improvements and I'll
separate these into one commit vs. one hotfix for `moreAvailable`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Reduce synchronization overhead for credit notifications
> --------------------------------------------------------
>
> Key: FLINK-10142
> URL: https://issues.apache.org/jira/browse/FLINK-10142
> Project: Flink
> Issue Type: Bug
> Components: Network
> Affects Versions: 1.5.2, 1.6.0, 1.7.0
> Reporter: Nico Kruber
> Assignee: Nico Kruber
> Priority: Major
> Labels: pull-request-available
>
> When credit-based flow control was introduced, we also added some checks and
> optimisations for uncommon code paths that make common code paths
> unnecessarily more expensive, e.g. checking whether a channel was released
> before forwarding a credit notification to Netty. Such checks would have to
> be confirmed by the Netty thread anyway and thus only add additional load for
> something that happens only once (per channel).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
