Github user Ethanlm commented on the issue:
https://github.com/apache/storm/pull/2270
@roshannaik Thanks for the review. I did some performance testing with
ACKing disabled on single Worker case:
Performance testing on ConstSpoutNullBoltTopo with ACKing disabled.
1. Config:
`topology.message.timeout: 300;
topology.max.spout.pending: 5000;
topology.acker.executors: 0`
2. 1 VM
3. Launched 1 workers, 1 spout task and 1 bolt task. ACKing disabled.
4. All experiments ran 300s.
5. For clarity, only show the outputs at 240s.
6. Numbers fluctuate slightly during the experiments.
Grouping | transsferred (messages) | transfer rate (message/s) |
spout_transferred | spout_acks | spout_throughput (acks/s)
-- | -- | -- | -- | -- | --
LocalityASG | 105218480 | 1753641 | 105218480 | 105218500 | 1753641
LocalOrShuffle(loadaware disabled) | 112664640 | 1877744 | 112664640 |
112664620 | 1877744
Shuffle (loadaware disabled) | 108752120 | 1812535 | 108752120 | 108752100
| 1812535
LocalOrShuffle | 79656420 | 1327607 | 79656420 | 79656420 | 1327607
Shuffle | 79809260 | 1330154 | 79809260 | 79809280 | 1330154
The results show that `ConcurrentMap` in `LocalityAwareShuffleGrouping`
introduces a very small overhead. We can also observe that `LoadAware` actually
introduces much overhead in this single Worker case. It's probably because of
`AtomicReference<Map<Integer,Load>>` data structure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
