[
https://issues.apache.org/jira/browse/STORM-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397638#comment-15397638
]
ASF GitHub Bot commented on STORM-2006:
---------------------------------------
Github user harshach commented on the issue:
https://github.com/apache/storm/pull/1595
@HeartSaVioR You improved the metrics collection and quality of those
metrics as well. This is immensely useful in debugging any topologies and
operationalizing the Storm it self. I am not sure if there any users (atleast I
haven't seen one) using the existing topology.metrics.consumer as its
performance is bad and can get the topology to a halt.
In light of these issues what you did here is will be extremely helpful
and learnings from here will help in the 2.0 changes as well.
Having said that this fix critical as we end up flooding the storage of
metrics with exponential number of metrics.
@ptgoetz "that contract is broken when this flag is set to true". Yes and
we are shipping this as false. So Unless user is consciously makes this as true
its not breaking anything. Sufficient documentation around this will help on
what to expect.
We also need to make a judgement call on what it will impact . With this
fix in users can go ahead store the metrics much more optimal fashion than what
was happening before.
IMHO this is not a backward-incompatible change and it will help users in
storing the metrics better.
I am still +1 on merging this patch.
"Btw, we should try avoiding to address current metrics feature and start
re-designing new metrics feature. To tell the truth, this feature is actually
closer to a hot-fix instead of improvement."
@HeartSaVioR agree. But this will get the users to start using the metrics
feature in storm. Before any of these changes all users depends on are UI
metrics and with these changes in they can more accurately look at whats going
on the cluster. Also they can add metrics into their topology. we definitely
should improve this feature for 2.0.
> Storm metrics feature improvement: support per-worker level metrics
> aggregation
> -------------------------------------------------------------------------------
>
> Key: STORM-2006
> URL: https://issues.apache.org/jira/browse/STORM-2006
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Storm provides per-task level metrics which could be huge when topology has a
> number of tasks.
> Task level metric is useful for determining load balance between tasks, but
> it doesn't need to be time-series fashion.
> Before introducing topology level component like TopologyMaster for JStorm,
> we can utilize SystemBolt to aggregate task level metrics to per-worker level
> metrics.
> We should provide options and this feature should be turned off by default to
> keep backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
