[
https://issues.apache.org/jira/browse/STORM-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394710#comment-15394710
]
ASF GitHub Bot commented on STORM-2006:
---------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1595#discussion_r72352692
--- Diff: conf/defaults.yaml ---
@@ -259,6 +259,10 @@ topology.disruptor.batch.size: 100
topology.disruptor.batch.timeout.millis: 1
topology.disable.loadaware: false
topology.state.checkpoint.interval.ms: 1000
+topology.metrics.aggregate.per.worker: false
--- End diff --
@harshach
Set this to 'true' breaks current metrics consumer implementations, since
it delegates aggregation to metrics consumer by passing list of values. We need
to guide the way how to aggregate these values to well-known metrics consumer
implementations.
Btw, we should try avoiding to address current metrics feature and start
re-designing new metrics feature. To tell the truth, this feature is actually
closer to a hot-fix instead of improvement.
Even I found the design issue while implementing this. Current metrics
feature doesn't allow us to aggregate the values, since we can't add the
metadata (which function we need to apply to aggregate) to metrics. Note that
we expand metrics from value of metric, so adding metadata to metric doesn't
make sense. Metric shouldn't use Object as value, especially some types are
intended to be expanded.
> Storm metrics feature improvement: support per-worker level metrics
> aggregation
> -------------------------------------------------------------------------------
>
> Key: STORM-2006
> URL: https://issues.apache.org/jira/browse/STORM-2006
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Storm provides per-task level metrics which could be huge when topology has a
> number of tasks.
> Task level metric is useful for determining load balance between tasks, but
> it doesn't need to be time-series fashion.
> Before introducing topology level component like TopologyMaster for JStorm,
> we can utilize SystemBolt to aggregate task level metrics to per-worker level
> metrics.
> We should provide options and this feature should be turned off by default to
> keep backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
