[ 
https://issues.apache.org/jira/browse/STORM-1007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14732014#comment-14732014
 ] 

ASF GitHub Bot commented on STORM-1007:
---------------------------------------

Github user wangli1426 commented on the pull request:

    https://github.com/apache/storm/pull/716#issuecomment-137968137
  
    Hi @HeartSaVioR,
    
    Thank you for your comment. I understand your concern, as 
```DisruptorQueue``` plays an important role in Storm.
    
    Following your suggestion, I have done performance test using 
[https://github.com/yahoo/storm-perf-test]. As expected, the results show that 
the new metrics do not introduce noticeable overhead.
    
    Environment:
    
    
      Factors | Configurations
    ------------ | -------------
    CPU | Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz
    RAM | 16GB
    DISK | 2TB
    OS | Ubuntu 14.02
    
    
    Parameters:
    
    ```
    --bolt 3 --name test -l 1 -n 1 --workers 4 --spout 3 --testTimeSec 900 -c 
topology.max.spout.pending=1092 --messageSize 10
    ```
    
    
    Result summary:
    
    master at 154e9ec55deb4eea8fca8554e4d3b224bf337834:
    Throughput in MB/s
    AVG: 7.974807415
    MIN : 7.878930664
    MAX: 8.095523041
    
    PR at f15ac94d259a96ac891f5243af29b0d953ea863d:
    Throughput in MB/s
    AVG: 7.977237486
    MIN : 7.844562531
    MAX: 8.251005809
    
    =======================================================
    
    The details of runs at master(154e9ec55deb4eea8fca8554e4d3b224bf337834)
    ```
    RUNNING     1       4       4       6       6       1441452581500   30000   
24470400        7.878930664
    RUNNING     1       4       4       6       6       1441452611500   30000   
24962680        7.935422262
    RUNNING     1       4       4       6       6       1441452641500   30000   
24765960        7.972886658
    RUNNING     1       4       4       6       6       1441452671500   30000   
24576620        7.982697093
    RUNNING     1       4       4       6       6       1441452701500   30000   
24901860        7.916088104
    RUNNING     1       4       4       6       6       1441452731500   30000   
25191040        8.008015951
    RUNNING     1       4       4       6       6       1441452761501   30001   
24791180        7.880641192
    RUNNING     1       4       4       6       6       1441452791500   29999   
25126800        7.987860867
    RUNNING     1       4       4       6       6       1441452821500   30000   
25071180        7.969913483
    RUNNING     1       4       4       6       6       1441452851500   30000   
25098540        7.978610992
    RUNNING     1       4       4       6       6       1441452881501   30001   
24966120        7.996251266
    RUNNING     1       4       4       6       6       1441452911500   29999   
24836340        8.095523041
    RUNNING     1       4       4       6       6       1441452941500   30000   
25004480        7.968710124
    RUNNING     1       4       4       6       6       1441452971500   30000   
25371360        8.065338135
    RUNNING     1       4       4       6       6       1441453001500   30000   
24972160        7.938435872
    RUNNING     1       4       4       6       6       1441453031500   30000   
25382020        8.068726858
    RUNNING     1       4       4       6       6       1441453061500   30000   
25021280        7.9540507
    RUNNING     1       4       4       6       6       1441453091500   30000   
25011120        7.950820923
    RUNNING     1       4       4       6       6       1441453121500   30000   
24690680        7.94895579
    RUNNING     1       4       4       6       6       1441453151500   30000   
24954060        7.972682037
    RUNNING     1       4       4       6       6       1441453181500   30000   
24899040        7.98519165
    RUNNING     1       4       4       6       6       1441453211501   30001   
24737460        7.963564633
    RUNNING     1       4       4       6       6       1441453241500   29999   
24996060        7.946298354
    RUNNING     1       4       4       6       6       1441453271500   30000   
24986100        7.942867279
    RUNNING     1       4       4       6       6       1441453301500   30000   
24906920        7.997696635
    RUNNING     1       4       4       6       6       1441453331500   30000   
25041520        7.960484823
    RUNNING     1       4       4       6       6       1441453361500   30000   
25121880        7.986030579
    RUNNING     1       4       4       6       6       1441453391500   30000   
25126380        7.98746109
    RUNNING     1       4       4       6       6       1441453421500   30000   
25043860        7.981228689
    RUNNING     1       4       4       6       6       1441453451501   30001   
24923920        8.022836695
    RUNNING     1       4       4       6       6       1441448928080   30000   
24357580        7.974807415
    RUNNING     1       4       4       6       6       1441448958080   30000   
24497360        7.878930664
    ```
    
    The details of runs at this PR(f15ac94d259a96ac891f5243af29b0d953ea863d)
    ```
    RUNNING     1       4       4       6       6       1441450879792   30001   
25703700        8.170713818
    RUNNING     1       4       4       6       6       1441450909791   29999   
25916500        8.238908104
    RUNNING     1       4       4       6       6       1441450939791   30000   
25955420        8.251005809
    RUNNING     1       4       4       6       6       1441450969791   30000   
25774120        8.193372091
    RUNNING     1       4       4       6       6       1441450999791   30000   
25701440        8.170267741
    RUNNING     1       4       4       6       6       1441451029791   30000   
25872800        8.224741618
    RUNNING     1       4       4       6       6       1441451059791   30000   
25357260        8.060855865
    RUNNING     1       4       4       6       6       1441451089791   30000   
24899640        7.915382385
    RUNNING     1       4       4       6       6       1441451119791   30000   
25170260        8.001410166
    RUNNING     1       4       4       6       6       1441451149792   30001   
25220280        8.017043861
    RUNNING     1       4       4       6       6       1441451179791   29999   
24963440        7.93592839
    RUNNING     1       4       4       6       6       1441451209791   30000   
24913380        7.919750214
    RUNNING     1       4       4       6       6       1441451239791   30000   
24676860        7.844562531
    RUNNING     1       4       4       6       6       1441451269791   30000   
25074360        7.970924377
    RUNNING     1       4       4       6       6       1441451299791   30000   
24991800        7.94467926
    RUNNING     1       4       4       6       6       1441451329791   30000   
24966800        7.936731974
    RUNNING     1       4       4       6       6       1441451359791   30000   
24789820        7.880471547
    RUNNING     1       4       4       6       6       1441451389791   30000   
24931320        7.925453186
    RUNNING     1       4       4       6       6       1441451419791   30000   
24845840        7.898279826
    RUNNING     1       4       4       6       6       1441451449791   30000   
24802200        7.884407043
    RUNNING     1       4       4       6       6       1441451479792   30001   
24923140        7.922588748
    RUNNING     1       4       4       6       6       1441451509791   29999   
24936560        7.927383183
    RUNNING     1       4       4       6       6       1441451539791   30000   
24810660        7.887096405
    RUNNING     1       4       4       6       6       1441451569792   30001   
25181340        8.004665581
    RUNNING     1       4       4       6       6       1441451599791   29999   
24692140        7.849681568
    RUNNING     1       4       4       6       6       1441451629791   30000   
24900600        7.915687561
    RUNNING     1       4       4       6       6       1441451659791   30000   
24762280        7.871716817
    RUNNING     1       4       4       6       6       1441451689791   30000   
24952080        7.932052612
    RUNNING     1       4       4       6       6       1441451719791   30000   
24917080        7.920926412
    RUNNING     1       4       4       6       6       1441451749791   30000   
25054500        7.964611053
    RUNNING     1       4       4       6       6       1441451779791   30000   
24871820        7.906538645
    ```
    
    Please let me know, if you have any question regarding to this tests or 
this PR.
    
    Thanks
    Li Wang


> Add more metrics to DisruptorQueue
> ----------------------------------
>
>                 Key: STORM-1007
>                 URL: https://issues.apache.org/jira/browse/STORM-1007
>             Project: Apache Storm
>          Issue Type: Bug
>            Reporter: Li Wang
>            Assignee: Li Wang
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> The metrics of the queues for each component are very helpful to reason about 
> the performance issues of a topology.
> For instance, for the applications with strong time constraint (e.g., threat 
> detection), if the elements in the input queue of a bolt have a long sojourn 
> time, it indicates that the user should increase the parallelism of that bolt 
> to reduce the processing delay.
> However, the metrics on the DisruptorQueue currently available are limited. 
> More useful metrics, such as average sojourn time and average throughput are 
> expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to