[
https://issues.apache.org/jira/browse/STORM-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15396393#comment-15396393
]
ASF GitHub Bot commented on STORM-2006:
---------------------------------------
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/1595#discussion_r72526172
--- Diff: conf/defaults.yaml ---
@@ -259,6 +259,10 @@ topology.disruptor.batch.size: 100
topology.disruptor.batch.timeout.millis: 1
topology.disable.loadaware: false
topology.state.checkpoint.interval.ms: 1000
+topology.metrics.aggregate.per.worker: false
--- End diff --
@ptgoetz
As you said this is just due to a design flaw of current metrics feature.
Return type of getValueAndReset (Object) just makes whole things opaque,
which is really bad.
Also we even didn't document a *implicit contract* that Map type of value
of metric should be expanded to multiple metrics. How to process metrics is
completely up to metrics consumer developer, and some metrics consumers even
don't expand Map type of value. I'm introducing similar thing from this change,
which I think it is still bad, but if you read the conversation, you may notice
that there's no other choice with current metrics.
@harshach @ptgoetz
This must be a last call of new feature/improvement of current metrics
feature. To tell the truth, I should never improve current metrics feature and
concentrate on new metrics feature, but I didn't and couldn't, because of
several reasons. My bad.
Actually my bad decision started from our progress of Storm 2.0.0 plan. For
the first time I'm just waiting for JStorm merger phase 2 so that JStorm
metrics feature to be applied. But we even didn't finish phase 1 in 6 months,
which is being said that it is expected to be finished in 1Q. So I was end up
with going on my own work, but have been considered about the backward
compatibility, and once I started considering that, I couldn't get out of it.
I'm still in doubt that we can finish JStorm merger by end of this year,
since phase 1 seems to be no progress on more than 1 month. If we would want to
concentrate port & merger works and make this finally happen, I can wait for
that. (Sure I'll participate the works.) But if it's out of our mind, we should
have the time to design and implement new metrics feature.
> Storm metrics feature improvement: support per-worker level metrics
> aggregation
> -------------------------------------------------------------------------------
>
> Key: STORM-2006
> URL: https://issues.apache.org/jira/browse/STORM-2006
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Storm provides per-task level metrics which could be huge when topology has a
> number of tasks.
> Task level metric is useful for determining load balance between tasks, but
> it doesn't need to be time-series fashion.
> Before introducing topology level component like TopologyMaster for JStorm,
> we can utilize SystemBolt to aggregate task level metrics to per-worker level
> metrics.
> We should provide options and this feature should be turned off by default to
> keep backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
