wsry commented on issue #9706: [FLINK-14118][runtime]Reduce the unnecessary flushing when there is no data available for flush. URL: https://github.com/apache/flink/pull/9706#issuecomment-539297211 @pnowojski Thanks for your comments. I have separated the benchmark commit into a new pr https://github.com/apache/flink/pull/9850 and some changes were made according to your suggestions. The latest results are as follows. Before the fix: ``` Benchmark Mode Cnt Score Error Units DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput thrpt 30 6014.192 ± 1090.448 ops/ms ``` After the fix: ``` Benchmark Mode Cnt Score Error Units DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput thrpt 30 18609.481 ± 715.554 ops/ms ``` The flowing are the whole output of jmh: Before the fix: ``` # JMH version: 1.19 # VM version: JDK 1.8.0_152, VM 25.152-b16 # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_152.jdk/Contents/Home/jre/bin/java # VM options: -javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=56029:/Applications/IntelliJ IDEA CE.app/Contents/bin -Dfile.encoding=UTF-8 -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.ssl # Warmup: 10 iterations, 1 s each # Measurement: 10 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.apache.flink.benchmark.DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput # Run progress: 0.00% complete, ETA 00:01:00 # Fork: 1 of 3 # Warmup Iteration 1: 2329.547 ops/ms # Warmup Iteration 2: 3446.658 ops/ms # Warmup Iteration 3: 4160.762 ops/ms # Warmup Iteration 4: 6923.136 ops/ms # Warmup Iteration 5: 6734.401 ops/ms # Warmup Iteration 6: 7778.430 ops/ms # Warmup Iteration 7: 4596.196 ops/ms # Warmup Iteration 8: 6886.789 ops/ms # Warmup Iteration 9: 7719.548 ops/ms # Warmup Iteration 10: 4906.588 ops/ms Iteration 1: 8042.346 ops/ms Iteration 2: 4754.322 ops/ms Iteration 3: 7352.920 ops/ms Iteration 4: 4806.341 ops/ms Iteration 5: 6639.124 ops/ms Iteration 6: 4271.681 ops/ms Iteration 7: 7125.093 ops/ms Iteration 8: 4391.322 ops/ms Iteration 9: 6600.411 ops/ms Iteration 10: 4769.296 ops/ms # Run progress: 33.33% complete, ETA 00:01:04 # Fork: 2 of 3 # Warmup Iteration 1: 2531.716 ops/ms # Warmup Iteration 2: 2608.277 ops/ms # Warmup Iteration 3: 4966.229 ops/ms # Warmup Iteration 4: 7160.782 ops/ms # Warmup Iteration 5: 6465.203 ops/ms # Warmup Iteration 6: 8318.341 ops/ms # Warmup Iteration 7: 4777.644 ops/ms # Warmup Iteration 8: 7914.236 ops/ms # Warmup Iteration 9: 7872.105 ops/ms # Warmup Iteration 10: 4641.590 ops/ms Iteration 1: 7304.282 ops/ms Iteration 2: 7551.630 ops/ms Iteration 3: 4965.887 ops/ms Iteration 4: 4680.523 ops/ms Iteration 5: 4527.468 ops/ms Iteration 6: 9564.202 ops/ms Iteration 7: 4383.187 ops/ms Iteration 8: 6944.958 ops/ms Iteration 9: 8225.094 ops/ms Iteration 10: 4742.338 ops/ms # Run progress: 66.67% complete, ETA 00:00:33 # Fork: 3 of 3 # Warmup Iteration 1: 2688.329 ops/ms # Warmup Iteration 2: 3664.663 ops/ms # Warmup Iteration 3: 9123.660 ops/ms # Warmup Iteration 4: 4310.853 ops/ms # Warmup Iteration 5: 4596.265 ops/ms # Warmup Iteration 6: 4632.639 ops/ms # Warmup Iteration 7: 4210.776 ops/ms # Warmup Iteration 8: 7430.278 ops/ms # Warmup Iteration 9: 4705.101 ops/ms # Warmup Iteration 10: 4601.005 ops/ms Iteration 1: 5874.745 ops/ms Iteration 2: 9210.923 ops/ms Iteration 3: 5271.976 ops/ms Iteration 4: 4686.587 ops/ms Iteration 5: 8222.316 ops/ms Iteration 6: 4521.516 ops/ms Iteration 7: 4545.242 ops/ms Iteration 8: 4558.030 ops/ms Iteration 9: 4638.766 ops/ms Iteration 10: 7253.224 ops/ms Result "org.apache.flink.benchmark.DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput": 6014.192 ±(99.9%) 1090.448 ops/ms [Average] (min, avg, max) = (4271.681, 6014.192, 9564.202), stdev = 1632.132 CI (99.9%): [4923.744, 7104.640] (assumes normal distribution) # Run complete. Total time: 00:01:37 Benchmark Mode Cnt Score Error Units DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput thrpt 30 6014.192 ± 1090.448 ops/ms Process finished with exit code 0 ``` After the fix: ``` # JMH version: 1.19 # VM version: JDK 1.8.0_152, VM 25.152-b16 # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_152.jdk/Contents/Home/jre/bin/java # VM options: -javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=49890:/Applications/IntelliJ IDEA CE.app/Contents/bin -Dfile.encoding=UTF-8 -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.ssl # Warmup: 10 iterations, 1 s each # Measurement: 10 iterations, 1 s each # Timeout: 10 min per iteration # Threads: 1 thread, will synchronize iterations # Benchmark mode: Throughput, ops/time # Benchmark: org.apache.flink.benchmark.DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput # Run progress: 0.00% complete, ETA 00:01:00 # Fork: 1 of 3 # Warmup Iteration 1: 3635.257 ops/ms # Warmup Iteration 2: 8143.488 ops/ms # Warmup Iteration 3: 13816.028 ops/ms # Warmup Iteration 4: 14774.969 ops/ms # Warmup Iteration 5: 17010.423 ops/ms # Warmup Iteration 6: 2180.684 ops/ms # Warmup Iteration 7: 17728.767 ops/ms # Warmup Iteration 8: 16845.906 ops/ms # Warmup Iteration 9: 17728.522 ops/ms # Warmup Iteration 10: 16360.496 ops/ms Iteration 1: 19476.796 ops/ms Iteration 2: 17029.085 ops/ms Iteration 3: 18558.875 ops/ms Iteration 4: 17294.359 ops/ms Iteration 5: 19501.712 ops/ms Iteration 6: 17772.873 ops/ms Iteration 7: 18087.300 ops/ms Iteration 8: 17944.233 ops/ms Iteration 9: 19895.783 ops/ms Iteration 10: 19683.808 ops/ms # Run progress: 33.33% complete, ETA 00:01:06 # Fork: 2 of 3 # Warmup Iteration 1: 3954.872 ops/ms # Warmup Iteration 2: 9800.140 ops/ms # Warmup Iteration 3: 11260.245 ops/ms # Warmup Iteration 4: 17343.418 ops/ms # Warmup Iteration 5: 19195.122 ops/ms # Warmup Iteration 6: 18282.445 ops/ms # Warmup Iteration 7: 17103.588 ops/ms # Warmup Iteration 8: 14640.616 ops/ms # Warmup Iteration 9: 17020.357 ops/ms # Warmup Iteration 10: 15240.485 ops/ms Iteration 1: 16878.102 ops/ms Iteration 2: 19786.107 ops/ms Iteration 3: 19660.232 ops/ms Iteration 4: 18026.600 ops/ms Iteration 5: 17958.209 ops/ms Iteration 6: 18059.793 ops/ms Iteration 7: 19261.274 ops/ms Iteration 8: 17403.674 ops/ms Iteration 9: 18244.055 ops/ms Iteration 10: 18591.080 ops/ms # Run progress: 66.67% complete, ETA 00:00:30 # Fork: 3 of 3 # Warmup Iteration 1: 4152.095 ops/ms # Warmup Iteration 2: 8226.688 ops/ms # Warmup Iteration 3: 15803.554 ops/ms # Warmup Iteration 4: 17761.849 ops/ms # Warmup Iteration 5: 19095.328 ops/ms # Warmup Iteration 6: 21287.998 ops/ms # Warmup Iteration 7: 17912.912 ops/ms # Warmup Iteration 8: 19136.508 ops/ms # Warmup Iteration 9: 19576.643 ops/ms # Warmup Iteration 10: 16607.639 ops/ms Iteration 1: 16720.474 ops/ms Iteration 2: 18854.064 ops/ms Iteration 3: 19570.923 ops/ms Iteration 4: 19189.055 ops/ms Iteration 5: 17770.440 ops/ms Iteration 6: 20835.050 ops/ms Iteration 7: 19607.576 ops/ms Iteration 8: 17573.196 ops/ms Iteration 9: 19074.172 ops/ms Iteration 10: 19975.534 ops/ms Result "org.apache.flink.benchmark.DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput": 18609.481 ±(99.9%) 715.554 ops/ms [Average] (min, avg, max) = (16720.474, 18609.481, 20835.050), stdev = 1071.007 CI (99.9%): [17893.928, 19325.035] (assumes normal distribution) # Run complete. Total time: 00:01:28 Benchmark Mode Cnt Score Error Units DataSkewStreamNetworkThroughputBenchmarkExecutor.networkSkewedThroughput thrpt 30 18609.481 ± 715.554 ops/ms Process finished with exit code 0 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
