Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/2502
Here's my test result: TVL with rate 85000 and max spout pending 5000.
spouts/splitters/counters are set to same as worker count.
Again, this is just to see if there's clear performance regression from TVL
which we have been using. Full analysis like [google doc in
STORM-2306](https://docs.google.com/document/d/1A5k41UjVFY8jZg01BHc1fFxmI0AcFz5jRiKhNjhOj7I/edit?usp=sharing)
would require so much efforts and resources (dedicated machines). If we could
update the numbers with latest master vs latest patch of STORM-2306 it would be
really great, but we should be OK if we postpone measuring numbers just before
releasing Storm 2.0.0 comparing with latest Storm 1.x release.
> 4 workers
>> master (ab7b4ca)
1.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,660.500 1,482.288 3,808.428 3,942.646
6.554 329.044 0
30 82,245.581 10.817 27.607 50.430
5.093 364.626 0
61 85,022.500 10.495 21.316 36.405
5.192 242.701 0
91 84,994.733 10.365 19.186 29.688
5.152 399.110 0
121 85,008.333 10.385 19.595 29.016
5.195 269.351 0
151 84,999.867 10.367 19.005 29.000
5.156 361.585 0
181 85,006.133 10.361 18.285 24.953
5.215 319.568 0
211 85,007.433 10.373 18.547 26.018
5.227 374.250 0
241 84,978.300 10.390 19.153 29.852
5.107 240.743 0
271 85,025.733 10.351 18.366 28.934
5.134 308.930 0
```
2.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 24,746.933 2,497.621 3,498.050 3,571.450
4.175 480.104 0
30 88,589.100 160.265 1,622.147 1,778.385
5.883 329.437 0
60 85,012.833 10.494 20.120 31.326
5.211 421.895 0
90 84,993.867 10.394 18.956 26.132
5.185 434.169 0
120 85,008.100 10.474 20.201 31.523
5.194 440.354 0
150 85,001.067 10.396 18.776 28.574
5.238 449.158 0
180 85,005.633 10.477 20.283 35.127
5.289 441.979 0
210 84,994.833 10.390 18.678 29.295
5.136 419.486 0
240 85,011.167 10.370 18.219 26.051
5.219 308.977 0
270 85,007.600 10.438 19.317 27.804
5.158 403.509 0
```
3.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,648.833 1,368.022 3,565.158 3,682.599
6.476 405.396 0
30 85,045.067 10.725 23.446 44.007
5.249 448.851 0
60 85,002.400 10.527 19.874 28.901
5.209 452.087 0
90 84,997.800 10.412 19.694 34.341
5.136 356.300 0
120 84,995.033 10.451 20.267 29.802
5.174 379.923 0
150 85,019.533 10.413 19.153 30.556
5.193 398.516 0
180 85,001.367 10.371 18.448 25.182
5.191 331.490 0
210 85,004.400 10.386 18.874 24.101
5.195 368.889 0
240 84,991.367 10.368 18.416 25.018
5.132 392.701 0
270 85,004.367 10.469 19.808 35.586
5.147 443.393 0
```
>> STORM-2306 (5b54809)
1.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 28,098.000 1,549.653 3,183.477 3,227.517
4.041 192.208 0
30 85,306.133 13.650 270.795 406.585
8.958 275.171 0
60 85,007.067 5.466 17.449 28.836
9.023 287.177 0
90 92,084.333 5.483 17.269 26.280
9.004 212.465 0
120 84,999.400 5.561 17.662 29.458
9.739 368.272 0
150 85,005.367 5.593 17.449 37.126
8.979 293.334 0
180 92,082.733 5.584 18.219 28.787
8.981 334.805 0
210 85,008.000 5.571 18.022 29.213
8.962 363.184 0
240 84,992.867 5.604 17.662 39.191
9.020 271.483 0
270 92,094.467 5.590 17.351 24.953
8.939 320.613 0
```
2.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,779.467 728.445 2,967.470 3,066.036
7.032 308.975 0
30 84,986.933 5.851 21.037 33.489
9.101 404.124 0
60 85,021.833 5.593 18.301 29.049
9.104 377.985 0
90 85,000.500 5.577 17.465 26.853
9.101 343.799 0
120 85,009.300 5.714 18.661 29.196
9.084 252.083 0
150 84,998.533 5.532 16.957 26.460
9.114 252.234 0
180 85,005.300 5.588 16.810 26.165
9.102 227.976 0
210 84,997.633 5.708 19.055 32.096
9.157 294.213 0
240 84,994.433 5.692 18.481 30.261
9.046 365.903 0
270 85,015.500 5.590 18.022 28.197
9.080 401.871 0
```
3.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 34,802.167 1,451.699 3,227.517 3,275.751
4.887 358.051 0
30 99,803.667 26.173 740.295 880.804
9.226 349.733 0
60 92,079.433 5.790 20.398 34.800
11.521 254.316 0
90 85,002.100 5.590 17.220 27.361
9.218 333.207 0
120 84,998.033 5.750 18.432 28.066
9.179 325.581 0
150 84,993.500 5.745 18.792 29.966
9.190 323.914 0
180 85,011.500 5.828 17.269 25.444
9.217 352.495 0
210 85,005.200 5.761 18.514 32.899
9.244 338.353 0
240 84,983.900 5.756 20.546 33.374
9.165 346.788 0
270 85,018.133 5.516 17.138 32.915
9.206 404.615 0
```
> 1 worker
>> master (ab7b4ca)
1.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,667.633 96.401 944.767 972.030
2.220 115.121 0
30 85,012.833 6.684 21.283 27.427
2.616 27.433 0
60 85,003.867 6.715 22.315 28.967
2.600 48.685 0
90 85,009.633 7.003 24.855 30.261
2.650 56.965 0
120 85,008.167 6.765 22.331 35.750
2.651 75.872 0
150 84,997.767 6.788 22.430 31.998
2.597 99.166 0
180 85,004.633 6.658 21.905 26.755
2.574 123.478 0
210 85,011.633 6.661 21.070 26.460
2.573 34.217 0
240 84,961.500 6.658 21.955 29.606
2.593 38.743 0
270 85,062.533 6.715 22.790 28.492
2.597 76.703 0
```
2.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,666.067 100.076 956.301 968.884
2.289 90.811 0
30 84,985.100 6.886 24.461 30.654
2.644 43.144 0
60 85,033.667 6.738 21.725 30.786
2.613 44.536 0
90 85,006.833 6.827 24.183 28.361
2.636 123.622 0
120 85,003.100 6.767 21.807 27.967
2.700 111.695 0
150 85,007.800 6.564 21.365 33.194
2.574 104.018 0
180 85,004.067 7.089 24.527 29.688
2.669 80.106 0
210 85,009.400 6.673 21.348 27.853
2.617 53.960 0
240 84,999.367 7.159 24.216 29.131
2.668 134.561 0
270 85,064.100 7.168 24.773 28.393
2.670 115.301 0
```
3.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,651.067 104.108 983.040 999.293
2.245 94.063 0
30 82,287.774 7.086 23.364 43.221
2.574 106.583 0
61 85,002.000 6.953 23.396 29.213
2.642 113.673 0
91 85,013.500 6.913 22.839 29.393
2.628 129.253 0
121 85,008.667 6.775 22.348 31.982
2.650 153.746 0
151 85,003.467 6.576 19.956 27.017
2.560 56.378 0
181 85,015.300 6.837 22.692 31.408
2.607 79.616 0
211 85,004.100 6.879 25.346 31.097
2.581 94.896 0
241 85,005.867 6.891 23.609 29.278
2.594 116.510 0
271 85,010.800 6.760 21.987 28.541
2.575 125.168 0
```
>> STORM-2306 (5b54809)
1.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,666.333 18.022 330.564 346.030
2.079 46.405 0
30 85,017.400 3.748 20.791 24.805
2.686 107.791 0
60 85,020.833 3.466 20.120 24.068
2.679 46.481 0
90 85,009.833 3.470 19.988 23.118
2.639 63.486 0
120 85,007.600 3.676 21.250 26.542
2.708 76.022 0
150 85,006.733 3.395 20.283 23.904
2.636 80.490 0
180 85,009.333 3.632 21.086 26.231
2.657 78.360 0
210 85,003.300 3.643 20.840 26.362
2.670 87.437 0
240 85,007.733 3.735 21.545 25.788
2.686 98.075 0
270 85,008.000 3.675 21.430 29.606
2.647 93.264 0
```
2.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,684.100 15.841 314.835 371.196
2.051 52.854 0
30 84,998.900 3.343 20.464 24.134
2.624 95.248 0
60 85,014.100 3.600 21.299 24.363
2.665 43.992 0
90 85,000.533 3.539 20.726 23.839
2.650 100.354 0
120 85,006.567 3.483 20.251 23.347
2.665 40.798 0
150 84,992.467 3.562 20.791 26.001
2.666 102.680 0
180 82,297.839 3.655 20.890 27.378
2.590 51.539 0
211 85,007.333 3.567 21.299 26.640
2.650 107.193 0
241 85,012.467 3.604 20.955 27.460
2.646 50.419 0
271 85,001.333 3.288 19.726 24.543
2.604 103.526 0
```
3.
```
(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms)
cores mem(MB) failed
0 56,680.200 14.695 299.893 340.263
2.096 100.239 0
30 85,012.833 4.007 21.512 27.427
2.719 67.345 0
60 85,003.900 3.921 22.233 31.228
2.709 43.670 0
90 85,009.467 3.881 22.249 26.722
2.683 23.378 0
120 85,027.600 3.955 21.840 25.772
2.717 108.810 0
150 85,009.300 4.049 22.479 36.405
2.724 80.150 0
180 85,005.933 3.381 20.283 23.478
2.639 43.224 0
210 85,007.733 3.798 21.053 24.199
2.717 115.814 0
240 85,007.800 4.275 22.266 26.608
2.751 106.709 0
270 85,005.867 3.507 20.644 24.297
2.700 59.038 0
```
In overall, this patch shows half of mean latency, whereas 99%ile/99.9%ile
of latencies are close to. It also shows this patch consumes more CPU. The gap
is fairly small for single-worker but becomes somewhat huge on 4 workers. I
might be wrong about testing so it would be really nice if someone also runs
the multi-workers test and shares the result.
Even my test result is not wrong I'm +1, because this patch introduces
better design of backpressure which should have been enabled by default but we
disabled. We could file some follow-up issues if there're something we should
address.
---
