[
https://issues.apache.org/jira/browse/STORM-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393515#comment-15393515
]
ASF GitHub Bot commented on STORM-2006:
---------------------------------------
GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1595
STORM-2006 Storm metrics feature improvement: support per-worker level
metrics aggregation
PR for 1.x : #1594
* SystemBolt handles task level metrics via two mode
* non-aggregate: same to previous, just applying expansion and pass to
MetricConsumerBolts
* aggregate: apply expansion, do aggregation, pass aggregated metrics to
MetricConsumerBolts
* all task level metrics should pass by SystemBolt within its worker
* it drops all connections between tasks and MetricsConsumerBolts
* only SystemBolts and MetricConsumerBolts will be connected
* move configurations: expandMapType and metricNameSeparator to global
* since SystemBolt needs to handle expansion when both worker level
aggregation and expandMapType are turned on
This could break #1445 and it's already broken. We ideally want to stop
adding new feature on 1.x and concentrate on 2.x but life is not easy.
I can volunteer to fix the conflict if this PR breaks #1445 in many places.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-2006
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1595.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1595
----
commit 8d1bc0b7ca2e50d45af3bc1bf41944925611910e
Author: Jungtaek Lim <[email protected]>
Date: 2016-07-25T13:43:02Z
STORM-2006 Storm metrics feature improvement: support per-worker level
metrics aggregation
* SystemBolt handles task level metrics via two mode
** non-aggregate: same to previous, just applying expansion and pass to
MetricConsumerBolts
** aggregate: apply expansion, do aggregation, pass aggregated metrics to
MetricConsumerBolts
* all task level metrics should pass by SystemBolt within its worker
** it drops all connections between tasks and MetricsConsumerBolts
** only SystemBolts and MetricConsumerBolts will be connected
* move configurations: expandMapType and metricNameSeparator to global
** since SystemBolt needs to handle expansion when both worker level
aggregation and expandMapType are turned on
----
> Storm metrics feature improvement: support per-worker level metrics
> aggregation
> -------------------------------------------------------------------------------
>
> Key: STORM-2006
> URL: https://issues.apache.org/jira/browse/STORM-2006
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Affects Versions: 1.1.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> Storm provides per-task level metrics which could be huge when topology has a
> number of tasks.
> Task level metric is useful for determining load balance between tasks, but
> it doesn't need to be time-series fashion.
> Before introducing topology level component like TopologyMaster for JStorm,
> we can utilize SystemBolt to aggregate task level metrics to per-worker level
> metrics.
> We should provide options and this feature should be turned off by default to
> keep backward compatibility.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
