Github user Ethanlm commented on the issue:
https://github.com/apache/storm/pull/2270
I did some experiments on ThroughputVsLatency (modified to add some
Configs) and the initial results seem similar among `shuffle`, `localOrShuffle`
and `localityAwareShuffle (LocalityASG)`.
Config:
`TOPOLOGY.MESSAGE.TIMEOUT: 300`
`TOPOLOGY_MAX_SPOUT_PENDING: 5000`
`LoadAware` is enabled by default
Env: Two openstack VM's, 8GB RAM, 4 VCPUs; 1Gbps Ethernet
Note:
1 All numbers in the tables are medians;
2 The numbers fluctuated slightly every time I ran the experiments. The
results shown below are sampled from repeated experiments.
Experiment1
Normal network connection.
This is trying to compare the performance in normal situation.
Rate | numWorkers | numSpout | numSplit(Bolt) | totalTime(min) |
ThroughputVsLatency | acked | acked/sec | failed | 99% | 99.90% | min | max |
mean | stddev | user | sys | gc | mem
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
-- | -- | -- | --
10 | 4 | 16 | 16 | 5 | LocalityASG | 2,204,700 | 73,490.00 | 0 |
69,726,109,695 | 72,410,464,255 | 51,271,172,096 | 74,423,730,175 |
59,100,119,611.86 | 5,520,976,189.22 | 176,570 | 33,110 | 19,981 | 1,729.69
10 | 4 | 16 | 16 | 5 | LocalOrShuffle | 2,082,000 | 67,333.33 | 0 |
66,102,231,039 | 66,538,438,655 | 50,096,766,976 | 66,806,874,111 |
57,631,698,540.71 | 3,643,782,002.52 | 172,500 | 30,060 | 20,803 | 1,175.51
10 | 4 | 16 | 16 | 5 | Shuffle | 2,067,640 | 68,812.00 | 0 |
72,611,790,847 | 73,752,641,535 | 50,298,093,568 | 74,557,947,903 |
62,488,041,809.45 | 5,480,336,956.59 | 184,020 | 29,220 | 23,905 | 1,363.61
3 | 4 | 16 | 16 | 5 | LocalityASG | 906,300 | 30,210.00 | 0 |
46,727,167 | 119,865,343 | 6,529,024 | 164,364,287 | 15,393,162.30 |
8,306,088.53 | 102,760 | 46,790 | 1,825 | 498.63
3 | 4 | 16 | 16 | 5 | LocalOrShuffle | 906,580 | 30,219.33 | 0 |
51,838,975 | 135,921,663 | 6,561,792 | 174,063,615 | 16,187,180.85 |
9,488,799.34 | 103,680 | 46,240 | 1,832 | 742.3
3 | 4 | 16 | 16 | 5 | Shuffle | 906,360 | 30,206.00 | 0 | 45,318,143 |
80,936,959 | 6,680,576 | 132,055,039 | 17,106,693.24 | 7,055,551.93 | 105,420 |
47,040 | 1,759 | 424.65
2 | 4 | 16 | 16 | 5 | LocalityASG | 605,040 | 20,168.00 | 0 |
31,965,183 | 49,840,127 | 5,500,928 | 96,534,527 | 13,949,066.47 | 4,487,713.39
| 94,370 | 48,930 | 1,253 | 433.45
2 | 4 | 16 | 16 | 5 | LocalOrShuffle | 604,640 | 20,154.67 | 0 |
32,161,791 | 87,621,631 | 5,525,504 | 124,518,399 | 14,090,342.83 |
5,387,101.60 | 92,780 | 47,170 | 1,239 | 356.32
2 | 4 | 16 | 16 | 5 | Shuffle | 604,840 | 20,151.33 | 0 | 33,406,975 |
70,909,951 | 4,399,104 | 98,697,215 | 14,103,326.48 | 5,074,175.87 | 90,870 |
47,230 | 1,193 | 396.6
1 | 4 | 16 | 16 | 5 | LocalityASG | 302,260 | 10,072.00 | 0 |
27,295,743 | 57,901,055 | 4,374,528 | 88,473,599 | 12,366,716.52 | 3,887,709.71
| 75,670 | 50,950 | 796 | 347.49
1 | 4 | 16 | 16 | 5 | LocalOrShuffle | 302,340 | 10,077.33 | 0 |
27,721,727 | 44,728,319 | 3,999,744 | 79,298,559 | 12,859,667.20 | 3,668,796.69
| 77,890 | 52,570 | 752 | 349.92
1 | 4 | 16 | 16 | 5 | Shuffle | 302,260 | 10,074.00 | 0 | 29,655,039 |
60,489,727 | 4,378,624 | 83,165,183 | 13,519,393.44 | 4,205,346.88 | 77,580 |
52,300 | 769 | 367.62
Experiment 2
Add 10ms latency on both VMs: `tc qdisc add dev eth0 root netem delay
10ms`, which means 20ms latency in total.
This is trying to simulate slow network connection
Rate | numWorkers | numSpout | numSplit(Bolt) | totalTime(min) |
ThroughputVsLatency | acked | acked/sec | failed | 99% | 99.90% | min | max |
mean | stddev | user | sys | gc | mem
-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- |
-- | -- | -- | --
10 | 4 | 16 | 16 | 5 | LocalityASG | 2,257,220 | 75,240.67 | 0 |
51,942,260,735 | 53,317,992,447 | 37,446,746,112 | 53,821,308,927 |
45,735,430,020.04 | 5,110,395,891.55 | 181,100 | 28,930 | 22,157 | 812.64
10 | 4 | 16 | 16 | 5 | LocalOrShuffle | 2,428,780 | 80,959.33 | 0 |
61,773,709,311 | 62,746,787,839 | 3,118,465,024 | 63,082,332,159 |
40,427,975,429.87 | 15,339,145,604.59 | 176,330 | 33,200 | 16,475 | 743.08
10 | 4 | 16 | 16 | 5 | Shuffle | 2,138,040 | 71,268.00 | 0 |
59,726,888,959 | 60,733,521,919 | 40,500,199,424 | 61,438,164,991 |
50,071,346,036.42 | 5,670,751,071.76 | 170,910 | 34,900 | 14,231 | 638.34
3 | 4 | 16 | 16 | 5 | LocalityASG | 908,140 | 30,271.33 | 0 |
330,301,439 | 383,254,527 | 7,000,064 | 416,808,959 | 48,647,996.71 |
55,147,277.57 | 105,160 | 44,800 | 1,975 | 579.67
3 | 4 | 16 | 16 | 5 | LocalOrShuffle | 902,680 | 30,089.33 | 0 |
341,049,343 | 383,254,527 | 6,946,816 | 414,973,951 | 51,019,429.45 |
58,367,846.16 | 104,980 | 47,240 | 1,895 | 642.05
3 | 4 | 16 | 16 | 5 |
