[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16613180#comment-16613180
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK closed pull request #6670: [FLINK-10301][network] extend 
StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/SerializingLongReceiver.java
 
b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/SerializingLongReceiver.java
index 580612ca03b..a93e9d2629d 100644
--- 
a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/SerializingLongReceiver.java
+++ 
b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/SerializingLongReceiver.java
@@ -40,6 +40,7 @@ public SerializingLongReceiver(InputGate inputGate, int 
expectedRepetitionsOfExp
});
}
 
+   @Override
protected void readRecords(long lastExpectedRecord) throws Exception {
LOG.debug("readRecords(lastExpectedRecord = {})", 
lastExpectedRecord);
final LongValue value = new LongValue();
diff --git 
a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 
b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
index 6b53488a5d3..bfaed437797 100644
--- 
a/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
+++ 
b/flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
@@ -68,11 +68,6 @@
  */
 public class StreamNetworkBenchmarkEnvironment {
 
-   private static final int BUFFER_SIZE =
-   
checkedDownCast(MemorySize.parse(TaskManagerOptions.MEMORY_SEGMENT_SIZE.defaultValue()).getBytes());
-
-   private static final int NUM_SLOTS_AND_THREADS = 1;
-
private static final InetAddress LOCAL_ADDRESS;
 
static {
@@ -96,6 +91,21 @@
 
protected ResultPartitionID[] partitionIds;
 
+   public void setUp(
+   int writers,
+   int channels,
+   boolean localMode,
+   int senderBufferPoolSize,
+   int receiverBufferPoolSize) throws Exception {
+   setUp(
+   writers,
+   channels,
+   localMode,
+   senderBufferPoolSize,
+   receiverBufferPoolSize,
+   new Configuration());
+   }
+
/**
 * Sets up the environment including buffer pools and netty threads.
 *
@@ -115,7 +125,8 @@ public void setUp(
int channels,
boolean localMode,
int senderBufferPoolSize,
-   int receiverBufferPoolSize) throws Exception {
+   int receiverBufferPoolSize,
+   Configuration config) throws Exception {
this.localMode = localMode;
this.channels = channels;
this.partitionIds = new ResultPartitionID[writers];
@@ -128,13 +139,13 @@ public void setUp(
 
ioManager = new IOManagerAsync();
 
-   senderEnv = createNettyNetworkEnvironment(senderBufferPoolSize);
+   senderEnv = createNettyNetworkEnvironment(senderBufferPoolSize, 
config);
senderEnv.start();
if (localMode && senderBufferPoolSize == 
receiverBufferPoolSize) {
receiverEnv = senderEnv;
}
else {
-   receiverEnv = 
createNettyNetworkEnvironment(receiverBufferPoolSize);
+   receiverEnv = 
createNettyNetworkEnvironment(receiverBufferPoolSize, config);
receiverEnv.start();
}
 
@@ -179,12 +190,25 @@ private void generatePartitionIds() throws Exception {
}
 
private NetworkEnvironment createNettyNetworkEnvironment(
-   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize) throws Exception {
+   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize, Configuration config) throws Exception {
+
+   int segmentSize =
+   checkedDownCast(
+   

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610749#comment-16610749
 ] 

ASF GitHub Bot commented on FLINK-10301:


pnowojski commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216700994
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -96,6 +91,21 @@
 
protected ResultPartitionID[] partitionIds;
 
+   public void setUp(
 
 Review comment:
   as you prefer


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610710#comment-16610710
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on issue #6670: [FLINK-10301][network] extend 
StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#issuecomment-420295302
 
 
   @pnowojski I assume the same thing :)
   - `BUFFER_SIZE` still uses the default (unless you set something specific in 
your `Configuration`)
   - NUM_SLOTS_AND_THREADS is still one (as long as no value for 
`NUM_TASK_SLOTS` is set or this is `1` or `-1`)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610701#comment-16610701
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216690643
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -96,6 +91,21 @@
 
protected ResultPartitionID[] partitionIds;
 
+   public void setUp(
 
 Review comment:
   not really...if you don't need a custom `Configuration` object, you can 
still use this as a shortcut(?)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610687#comment-16610687
 ] 

ASF GitHub Bot commented on FLINK-10301:


pnowojski commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216686306
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -96,6 +91,21 @@
 
protected ResultPartitionID[] partitionIds;
 
+   public void setUp(
 
 Review comment:
   `@Depreacted`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610317#comment-16610317
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216595121
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -179,12 +192,15 @@ private void generatePartitionIds() throws Exception {
}
 
private NetworkEnvironment createNettyNetworkEnvironment(
-   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize) throws Exception {
+   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize, Configuration config) throws Exception {
+
+   int segmentSize =
+   
checkedDownCast(MemorySize.parse(config.getString(TaskManagerOptions.MEMORY_SEGMENT_SIZE)).getBytes());
 
-   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, BUFFER_SIZE);
+   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, segmentSize);
 
final NettyConnectionManager nettyConnectionManager = new 
NettyConnectionManager(
-   new NettyConfig(LOCAL_ADDRESS, 0, BUFFER_SIZE, 
NUM_SLOTS_AND_THREADS, new Configuration()));
 
 Review comment:
   good catch - actually having 1 Netty thread per server/client might 
eventually become the bottleneck depending on the benchmark...I'll use the 
property from the `config` using `TaskManagerOptions.NUM_TASK_SLOTS` as is done 
in Flink


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610315#comment-16610315
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216595121
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -179,12 +192,15 @@ private void generatePartitionIds() throws Exception {
}
 
private NetworkEnvironment createNettyNetworkEnvironment(
-   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize) throws Exception {
+   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize, Configuration config) throws Exception {
+
+   int segmentSize =
+   
checkedDownCast(MemorySize.parse(config.getString(TaskManagerOptions.MEMORY_SEGMENT_SIZE)).getBytes());
 
-   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, BUFFER_SIZE);
+   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, segmentSize);
 
final NettyConnectionManager nettyConnectionManager = new 
NettyConnectionManager(
-   new NettyConfig(LOCAL_ADDRESS, 0, BUFFER_SIZE, 
NUM_SLOTS_AND_THREADS, new Configuration()));
 
 Review comment:
   good catch - actually having 1 Netty thread per server/client might 
eventually become the bottleneck depending on the benchmark...I'll use the 
property from the `config`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610294#comment-16610294
 ] 

ASF GitHub Bot commented on FLINK-10301:


zhijiangW commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216565959
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -179,12 +192,15 @@ private void generatePartitionIds() throws Exception {
}
 
private NetworkEnvironment createNettyNetworkEnvironment(
-   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize) throws Exception {
+   @SuppressWarnings("SameParameterValue") int 
bufferPoolSize, Configuration config) throws Exception {
+
+   int segmentSize =
+   
checkedDownCast(MemorySize.parse(config.getString(TaskManagerOptions.MEMORY_SEGMENT_SIZE)).getBytes());
 
-   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, BUFFER_SIZE);
+   final NetworkBufferPool bufferPool = new 
NetworkBufferPool(bufferPoolSize, segmentSize);
 
final NettyConnectionManager nettyConnectionManager = new 
NettyConnectionManager(
-   new NettyConfig(LOCAL_ADDRESS, 0, BUFFER_SIZE, 
NUM_SLOTS_AND_THREADS, new Configuration()));
 
 Review comment:
   Do we also need get the value of `NUM_SLOTS_AND_THREADS` from this `config`?
   I know the current usages of `NUM_SLOTS_AND_THREADS` are only for 
`NUM_ARENAS` and `NUM_THREADS_SERVER` which both have separate parameters in 
`NettyConfig`.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610235#comment-16610235
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on issue #6670: [FLINK-10301][network] extend 
StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#issuecomment-420183855
 
 
   @zhijiangW I think, I already did by creating the overloaded 
`StreamNetworkPointToPointBenchmark#setUp(long, Configuration)` method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610190#comment-16610190
 ] 

ASF GitHub Bot commented on FLINK-10301:


zhijiangW commented on issue #6670: [FLINK-10301][network] extend 
StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#issuecomment-420172687
 
 
   @NicoK  Thank you for the changes and it can help for adjusting netty 
related parameters during testing.
   BTY, Would you plan to make the similar changes for 
`StreamNetworkPointToPointBenchmark` ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608915#comment-16608915
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r216249203
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -115,7 +115,8 @@ public void setUp(
int channels,
boolean localMode,
int senderBufferPoolSize,
-   int receiverBufferPoolSize) throws Exception {
+   int receiverBufferPoolSize,
+   Configuration config) throws Exception {
 
 Review comment:
   sure, why not (I'm backporting this patch to Flink 1.5+ anyway, but not all 
tags)
   
   -> did the same for the other `setUp()` methods


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607024#comment-16607024
 ] 

ASF GitHub Bot commented on FLINK-10301:


zentol commented on a change in pull request #6670: [FLINK-10301][network] 
extend StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670#discussion_r215934568
 
 

 ##
 File path: 
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/io/benchmark/StreamNetworkBenchmarkEnvironment.java
 ##
 @@ -115,7 +115,8 @@ public void setUp(
int channels,
boolean localMode,
int senderBufferPoolSize,
-   int receiverBufferPoolSize) throws Exception {
+   int receiverBufferPoolSize,
+   Configuration config) throws Exception {
 
 Review comment:
   How about overloading this method instead with a variant that also accepts a 
configuration? Then it would be easier to create benchmarks across different 
flink versions.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (FLINK-10301) Allow a custom Configuration in StreamNetworkBenchmarkEnvironment

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/FLINK-10301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16607016#comment-16607016
 ] 

ASF GitHub Bot commented on FLINK-10301:


NicoK opened a new pull request #6670: [FLINK-10301][network] extend 
StreamNetworkBenchmarkEnvironment to allow custom Configuration instances
URL: https://github.com/apache/flink/pull/6670
 
 
   ## What is the purpose of the change
   
   `StreamNetworkBenchmarkEnvironment` currently instantiates its `NettyConfig` 
with `new Configuration()` but for testing and benchmarking with different 
options, it would be nice to allow the user to provide a custom instance 
instead.
   
   ## Brief change log
   
   - extend `StreamNetworkBenchmarkEnvironment#createNettyNetworkEnvironment` 
and related methods with a `Configuration` parameter
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as 
`StreamNetworkPointToPointBenchmarkTest` and 
`StreamNetworkThroughputBenchmarkTest`.
   
   ## Does this pull request potentially affect one of the following parts:
   
 - Dependencies (does it add or upgrade a dependency): **no**
 - The public API, i.e., is any changed class annotated with 
`@Public(Evolving)`: **no**
 - The serializers: **no**
 - The runtime per-record code paths (performance sensitive): **no**
 - Anything that affects deployment or recovery: JobManager (and its 
components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
 - The S3 file system connector: **no**
   
   ## Documentation
   
 - Does this pull request introduce a new feature? **no**
 - If yes, how is the feature documented? **not applicable**
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Allow a custom Configuration in StreamNetworkBenchmarkEnvironment
> -
>
> Key: FLINK-10301
> URL: https://issues.apache.org/jira/browse/FLINK-10301
> Project: Flink
>  Issue Type: Improvement
>  Components: Network, Tests
>Affects Versions: 1.7.0
>Reporter: Nico Kruber
>Assignee: Nico Kruber
>Priority: Major
>  Labels: pull-request-available
>
> {{StreamNetworkBenchmarkEnvironment}} currently instantiates its 
> {{NettyConfig}} with {{new Configuration()}} but for testing with different 
> options, it would be nice to allow the user to provide a custom instance 
> instead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)