[GitHub] [flink] guoweiM commented on a change in pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
guoweiM commented on a change in pull request #15652:
URL: https://github.com/apache/flink/pull/15652#discussion_r615332364
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
##
@@ -160,18 +160,12 @@ public void setup() throws IOException {
}
int numRequiredBuffer = bufferPool.getNumberOfRequiredMemorySegments();
-String errorMessage =
-String.format(
-"Too few sort buffers, please increase %s to a larger
value (more than %d).",
-
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS,
-2 * expectedWriteBuffers);
-if (numRequiredBuffer < 2 * expectedWriteBuffers) {
-LOG.warn(errorMessage);
-}
-
int numWriteBuffers = Math.min(numRequiredBuffer / 2,
expectedWriteBuffers);
if (numWriteBuffers < 1) {
-throw new IOException(errorMessage);
+throw new IOException(
+String.format(
+"Too few sort buffers, please increase %s.",
+
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS));
Review comment:
Thanks for explaining.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [flink] guoweiM commented on a change in pull request #15652: [FLINK-22305][network] Improve log messages of sort-merge blocking shuffle
guoweiM commented on a change in pull request #15652:
URL: https://github.com/apache/flink/pull/15652#discussion_r615328699
##
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/SortMergeResultPartition.java
##
@@ -160,18 +160,12 @@ public void setup() throws IOException {
}
int numRequiredBuffer = bufferPool.getNumberOfRequiredMemorySegments();
-String errorMessage =
-String.format(
-"Too few sort buffers, please increase %s to a larger
value (more than %d).",
-
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS,
-2 * expectedWriteBuffers);
-if (numRequiredBuffer < 2 * expectedWriteBuffers) {
-LOG.warn(errorMessage);
-}
-
int numWriteBuffers = Math.min(numRequiredBuffer / 2,
expectedWriteBuffers);
if (numWriteBuffers < 1) {
-throw new IOException(errorMessage);
+throw new IOException(
+String.format(
+"Too few sort buffers, please increase %s.",
+
NettyShuffleEnvironmentOptions.NETWORK_SORT_SHUFFLE_MIN_BUFFERS));
Review comment:
I think this it might not be clearer than the before. The old message
tells user increase to what size of the network buffer but the new message does
not tell the size of network buffer that user should increase to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
