[
https://issues.apache.org/jira/browse/STORM-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399234#comment-15399234
]
ASF GitHub Bot commented on STORM-2006:
---------------------------------------
Github user abhishekagarwal87 commented on the issue:
https://github.com/apache/storm/pull/1595
@HeartSaVioR - Thanks for putting in efforts on improving metrics. The
approach of passing metrics via system bolt looks agreeable to me. Though I
would still prefer "Two types of metric consumers approach".
Secondly regarding aggregation, component level aggregation is much more
preferred. That is what we do in external systems. Simply for the reason that
`total number of messages processed by component A` is more useful and
actionable information than `total number of messages processed on this machine
and port`. If the component information is being let go during aggregation, I
don't think it will be used much. At least I wouldn't in our production system.
Load among tasks is usually homogenous and uniform (law of large numbers)
but same is not true for different components which execute different code
altogether. I think just aggregating per worker+component combination would
also reduce the amount of metrics being emitted and still keep the metrics
useful.
Thirdly, the assumption that a metric being declared with 60 second update
interval will actually come in 60 second intervals is not correct. They can
come with 120 seconds apart or they can also come 5 seconds apart (depending on
how fast or slow the task is).
> Storm metrics feature improvement: support per-worker level metrics
> aggregation
> -------------------------------------------------------------------------------
>
> Key: STORM-2006
> URL: https://issues.apache.org/jira/browse/STORM-2006
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Storm provides per-task level metrics which could be huge when topology has a
> number of tasks.
> Task level metric is useful for determining load balance between tasks, but
> it doesn't need to be time-series fashion.
> Before introducing topology level component like TopologyMaster for JStorm,
> we can utilize SystemBolt to aggregate task level metrics to per-worker level
> metrics.
> We should provide options and this feature should be turned off by default to
> keep backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
