[
https://issues.apache.org/jira/browse/STORM-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14732014#comment-14732014
]
ASF GitHub Bot commented on STORM-1007:
---------------------------------------
Github user wangli1426 commented on the pull request:
https://github.com/apache/storm/pull/716#issuecomment-137968137
Hi @HeartSaVioR,
Thank you for your comment. I understand your concern, as
```DisruptorQueue``` plays an important role in Storm.
Following your suggestion, I have done performance test using
[https://github.com/yahoo/storm-perf-test]. As expected, the results show that
the new metrics do not introduce noticeable overhead.
Environment:
Factors | Configurations
------------ | -------------
CPU | Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz
RAM | 16GB
DISK | 2TB
OS | Ubuntu 14.02
Parameters:
```
--bolt 3 --name test -l 1 -n 1 --workers 4 --spout 3 --testTimeSec 900 -c
topology.max.spout.pending=1092 --messageSize 10
```
Result summary:
master at 154e9ec55deb4eea8fca8554e4d3b224bf337834:
Throughput in MB/s
AVG: 7.974807415
MIN : 7.878930664
MAX: 8.095523041
PR at f15ac94d259a96ac891f5243af29b0d953ea863d:
Throughput in MB/s
AVG: 7.977237486
MIN : 7.844562531
MAX: 8.251005809
=======================================================
The details of runs at master(154e9ec55deb4eea8fca8554e4d3b224bf337834)
```
RUNNING 1 4 4 6 6 1441452581500 30000
24470400 7.878930664
RUNNING 1 4 4 6 6 1441452611500 30000
24962680 7.935422262
RUNNING 1 4 4 6 6 1441452641500 30000
24765960 7.972886658
RUNNING 1 4 4 6 6 1441452671500 30000
24576620 7.982697093
RUNNING 1 4 4 6 6 1441452701500 30000
24901860 7.916088104
RUNNING 1 4 4 6 6 1441452731500 30000
25191040 8.008015951
RUNNING 1 4 4 6 6 1441452761501 30001
24791180 7.880641192
RUNNING 1 4 4 6 6 1441452791500 29999
25126800 7.987860867
RUNNING 1 4 4 6 6 1441452821500 30000
25071180 7.969913483
RUNNING 1 4 4 6 6 1441452851500 30000
25098540 7.978610992
RUNNING 1 4 4 6 6 1441452881501 30001
24966120 7.996251266
RUNNING 1 4 4 6 6 1441452911500 29999
24836340 8.095523041
RUNNING 1 4 4 6 6 1441452941500 30000
25004480 7.968710124
RUNNING 1 4 4 6 6 1441452971500 30000
25371360 8.065338135
RUNNING 1 4 4 6 6 1441453001500 30000
24972160 7.938435872
RUNNING 1 4 4 6 6 1441453031500 30000
25382020 8.068726858
RUNNING 1 4 4 6 6 1441453061500 30000
25021280 7.9540507
RUNNING 1 4 4 6 6 1441453091500 30000
25011120 7.950820923
RUNNING 1 4 4 6 6 1441453121500 30000
24690680 7.94895579
RUNNING 1 4 4 6 6 1441453151500 30000
24954060 7.972682037
RUNNING 1 4 4 6 6 1441453181500 30000
24899040 7.98519165
RUNNING 1 4 4 6 6 1441453211501 30001
24737460 7.963564633
RUNNING 1 4 4 6 6 1441453241500 29999
24996060 7.946298354
RUNNING 1 4 4 6 6 1441453271500 30000
24986100 7.942867279
RUNNING 1 4 4 6 6 1441453301500 30000
24906920 7.997696635
RUNNING 1 4 4 6 6 1441453331500 30000
25041520 7.960484823
RUNNING 1 4 4 6 6 1441453361500 30000
25121880 7.986030579
RUNNING 1 4 4 6 6 1441453391500 30000
25126380 7.98746109
RUNNING 1 4 4 6 6 1441453421500 30000
25043860 7.981228689
RUNNING 1 4 4 6 6 1441453451501 30001
24923920 8.022836695
RUNNING 1 4 4 6 6 1441448928080 30000
24357580 7.974807415
RUNNING 1 4 4 6 6 1441448958080 30000
24497360 7.878930664
```
The details of runs at this PR(f15ac94d259a96ac891f5243af29b0d953ea863d)
```
RUNNING 1 4 4 6 6 1441450879792 30001
25703700 8.170713818
RUNNING 1 4 4 6 6 1441450909791 29999
25916500 8.238908104
RUNNING 1 4 4 6 6 1441450939791 30000
25955420 8.251005809
RUNNING 1 4 4 6 6 1441450969791 30000
25774120 8.193372091
RUNNING 1 4 4 6 6 1441450999791 30000
25701440 8.170267741
RUNNING 1 4 4 6 6 1441451029791 30000
25872800 8.224741618
RUNNING 1 4 4 6 6 1441451059791 30000
25357260 8.060855865
RUNNING 1 4 4 6 6 1441451089791 30000
24899640 7.915382385
RUNNING 1 4 4 6 6 1441451119791 30000
25170260 8.001410166
RUNNING 1 4 4 6 6 1441451149792 30001
25220280 8.017043861
RUNNING 1 4 4 6 6 1441451179791 29999
24963440 7.93592839
RUNNING 1 4 4 6 6 1441451209791 30000
24913380 7.919750214
RUNNING 1 4 4 6 6 1441451239791 30000
24676860 7.844562531
RUNNING 1 4 4 6 6 1441451269791 30000
25074360 7.970924377
RUNNING 1 4 4 6 6 1441451299791 30000
24991800 7.94467926
RUNNING 1 4 4 6 6 1441451329791 30000
24966800 7.936731974
RUNNING 1 4 4 6 6 1441451359791 30000
24789820 7.880471547
RUNNING 1 4 4 6 6 1441451389791 30000
24931320 7.925453186
RUNNING 1 4 4 6 6 1441451419791 30000
24845840 7.898279826
RUNNING 1 4 4 6 6 1441451449791 30000
24802200 7.884407043
RUNNING 1 4 4 6 6 1441451479792 30001
24923140 7.922588748
RUNNING 1 4 4 6 6 1441451509791 29999
24936560 7.927383183
RUNNING 1 4 4 6 6 1441451539791 30000
24810660 7.887096405
RUNNING 1 4 4 6 6 1441451569792 30001
25181340 8.004665581
RUNNING 1 4 4 6 6 1441451599791 29999
24692140 7.849681568
RUNNING 1 4 4 6 6 1441451629791 30000
24900600 7.915687561
RUNNING 1 4 4 6 6 1441451659791 30000
24762280 7.871716817
RUNNING 1 4 4 6 6 1441451689791 30000
24952080 7.932052612
RUNNING 1 4 4 6 6 1441451719791 30000
24917080 7.920926412
RUNNING 1 4 4 6 6 1441451749791 30000
25054500 7.964611053
RUNNING 1 4 4 6 6 1441451779791 30000
24871820 7.906538645
```
Please let me know, if you have any question regarding to this tests or
this PR.
Thanks
Li Wang
> Add more metrics to DisruptorQueue
> ----------------------------------
>
> Key: STORM-1007
> URL: https://issues.apache.org/jira/browse/STORM-1007
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Li Wang
> Assignee: Li Wang
> Original Estimate: 672h
> Remaining Estimate: 672h
>
> The metrics of the queues for each component are very helpful to reason about
> the performance issues of a topology.
> For instance, for the applications with strong time constraint (e.g., threat
> detection), if the elements in the input queue of a bolt have a long sojourn
> time, it indicates that the user should increase the parallelism of that bolt
> to reduce the processing delay.
> However, the metrics on the DisruptorQueue currently available are limited.
> More useful metrics, such as average sojourn time and average throughput are
> expected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
