[ https://issues.apache.org/jira/browse/STORM-3205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614579#comment-16614579 ]
Roshan Naik commented on STORM-3205: ------------------------------------ *Raw TVL readings:* +MASTER: reading 1+ {code} start_time(s) end_time(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms) cores mem(MB) 0 30 363,105.633 552.793 844.104 853.017 3.804 174.941 30 60 447,122.200 2,142.178 5,695.865 5,804.917 4.550 51.488 60 90 416,901.400 9,418.863 13,019.120 13,086.228 4.414 155.132 90 120 420,867.467 16,801.866 20,099.105 20,149.436 4.352 100.409 120 150 429,340.700 23,451.448 26,575.110 26,742.882 4.412 105.838 150 180 427,491.233 30,118.689 33,386.660 33,436.991 4.417 145.061 180 210 410,376.700 37,208.048 41,003.516 41,070.625 4.390 139.888 210 240 420,193.767 44,735.209 48,083.501 48,184.164 4.392 97.299 240 270 424,774.967 51,567.712 54,962.160 55,029.268 4.412 127.620 {code} +MASTER: reading2+ {code} start_time(s) end_time(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms) cores mem(MB) 0 30 352,283.467 666.673 988.807 996.147 3.833 131.411 30 60 452,429.567 2,457.883 5,981.078 6,106.907 4.825 128.944 60 90 389,890.233 10,536.345 14,738.784 14,856.225 4.246 145.399 90 120 399,513.033 19,105.363 22,917.677 23,068.672 4.138 135.547 120 150 412,216.733 26,820.408 30,450.647 30,601.642 4.195 55.722 150 180 415,783.600 34,411.969 37,882.954 37,950.063 4.235 95.921 180 210 428,794.100 41,234.477 44,493.177 44,560.286 4.383 71.357 210 240 427,405.133 47,840.604 51,204.063 51,271.172 4.349 112.849 {code} +STORM-3205 : TVL reading1+ {code} start_time(s) end_time(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms) cores mem(MB) 0 30 365,140.233 389.399 851.444 865.599 3.733 112.723 30 60 551,828.200 2.096 58.917 109.838 4.593 68.210 60 90 550,123.200 0.658 4.268 6.423 4.433 107.547 90 120 550,244.433 0.567 1.696 6.160 4.146 113.571 120 150 550,271.567 0.547 1.245 3.652 4.170 144.445 150 180 550,279.300 1.077 18.792 25.756 4.229 112.253 180 210 550,234.733 0.529 1.136 1.579 4.069 136.361 210 240 550,241.433 0.530 1.150 1.587 4.087 160.657 {code} +STORM-3205 : TVL reading2+ {code} start_time(s) end_time(s) rate(tuple/s) mean(ms) 99%ile(ms) 99.9%ile(ms) cores mem(MB) 0 30 366,992.967 280.818 569.377 575.144 3.693 155.890 30 60 550,076.200 0.556 1.385 5.751 4.023 163.471 60 91 532,387.032 0.555 1.333 5.321 3.958 142.208 91 121 550,137.600 0.545 1.209 4.522 4.024 120.298 121 151 550,156.633 0.569 1.647 5.382 4.210 87.505 151 181 550,222.900 0.574 1.808 5.730 4.176 169.807 181 211 550,226.400 0.535 1.202 1.917 4.133 154.749 211 241 550,156.300 0.530 1.150 1.644 4.052 117.847 {code} > Optimization in TuplImpl > ------------------------ > > Key: STORM-3205 > URL: https://issues.apache.org/jira/browse/STORM-3205 > Project: Apache Storm > Issue Type: Improvement > Components: storm-client > Affects Versions: 2.0.0 > Reporter: Roshan Naik > Assignee: Roshan Naik > Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Wrapping {{TuplImpl.values}} with Collections.unmodifiableList() turns out be > very expensive. Its intention is obviously to check and prevent accidental > tweaking TuplImpl once created. Given the high cost, if needed, we can limit > this extra checking mechanism in debug/dev mode. Being in the critical path > it means several thousand/million additional allocations per second of the > List wrapper object .... proportional to the number of bolt/spout instances. > *TVL :* > | |throughput (k/sec)|cores|mem (mb)| > |*master (#f5a410ba3)*|412 |4.26|103| > |*storm-3205*|547 (+33%)|4.09|132| > {{+cmd:+ bin/storm jar topos/storm-loadgen-2.0.0-SNAPSHOT.jar > org.apache.storm.loadgen.ThroughputVsLatency *--rate 550000* --spouts 1 > --splitters 3 --counters 2 -c topology.acker.executors=0}} > *+ConstSpoutIdentityBoltNullBolt :+* > | |throughput| > |*master*|4.25 mill/sec| > |*storm-3205*|5.4 mill/sec (+27%)| > +cmd+: {{bin/storm jar topos/storm-perf-2.0.0-SNAPSHOT.jar > org.apache.storm.perf.ConstSpoutIdBoltNullBoltTopo -c > topology.acker.executors=0 -c topology.producer.batch.size=1000 400}} > *Note:* The perf gains are more evident when operating at high thoughputs w/o > backpressure occurring (i.e. some bolts have not yet become a bottleneck) -- This message was sent by Atlassian JIRA (v7.6.3#76005)