[jira] [Created] (HADOOP-14308) GraphiteSink gives up reconnecting to the graphite server
Damien Claveau created HADOOP-14308: --- Summary: GraphiteSink gives up reconnecting to the graphite server Key: HADOOP-14308 URL: https://issues.apache.org/jira/browse/HADOOP-14308 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.7.3, 2.7.2, 2.7.1, 2.7.0 Reporter: Damien Claveau Assignee: Damien Claveau Priority: Minor Since https://issues.apache.org/jira/browse/HADOOP-11400 the GraphiteSink now handles the reconnection, but it gives up silently after 5 failed attempts. As discussed here https://issues.apache.org/jira/browse/HADOOP-11400?focusedCommentId=14977496=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14977496, the sink should better throw an exception then rely on the MetricsSinkAdapter logic to retry with the configured retryDelay and retryBackoff parameters. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14300) GraphiteSink reports metrics containing undesirable whitespaces
[
https://issues.apache.org/jira/browse/HADOOP-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968778#comment-15968778
]
Damien Claveau commented on HADOOP-14300:
-
Hi Wei-Chiu, thank you for the precommit check.
I don't think the failed junit tests on hadoop.security.TestKDiag can be
related to the patch.
> GraphiteSink reports metrics containing undesirable whitespaces
> ---
>
> Key: HADOOP-14300
> URL: https://issues.apache.org/jira/browse/HADOOP-14300
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.7.2, 2.7.3
>Reporter: Damien Claveau
>Assignee: Damien Claveau
>Priority: Trivial
> Attachments: HADOOP-14300, HADOOP-14300.001.patch
>
>
> According to the Graphite documentation, the data sent must be in the
> following format:
> But some tag values that are flattened in the metricpath string sometimes
> include whitespaces and violate the carbon/graphite format.
> For example, the dirpath enumeration from the datanode :
> FSDatasetState.org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.Context=FSDatasetState.StorageInfo=FSDataset{dirpath='[/data/1/hdfs/data/current,
> /data/2/hdfs/data/current]'}.Hostname=worker1.Capacity 78436466688 1491986609
> Currently, none of the aggregators available in the Graphite ecosystem can
> handle these malformed metrics. These whitespaces should be replaced by
> underscores.
> I will be happy to attach a small patch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14307) GraphiteSink can specify which tags to export into the metrics prefix
Damien Claveau created HADOOP-14307: --- Summary: GraphiteSink can specify which tags to export into the metrics prefix Key: HADOOP-14307 URL: https://issues.apache.org/jira/browse/HADOOP-14307 Project: Hadoop Common Issue Type: Improvement Components: metrics Affects Versions: 2.7.3, 2.7.2, 2.7.1, 2.7.0 Reporter: Damien Claveau Assignee: Damien Claveau Priority: Minor Fix For: 2.7.3, 2.7.2, 2.7.1, 2.7.0 This Jira is a feature proposal to add the ability in GraphiteSink to specify which Tag (Name/Value) pairs are to be flattened in the metric prefix. The motivation for this is that currently, all the tags are included in the prefix like this : graphite_prefix.Context=$context.Process=$process.Tag1=$value1.Tag2=$value2..Tag9=$value9.metric This requires a bunch of rewriting rules (and complex regexp) in the metric aggregation servers (carbon-relay, carbon-aggregator, ...) . The feature would be exactly the same as the solution implemented in the GangliaSink : https://issues.apache.org/jira/browse/HADOOP-7507 See also the commit : https://github.com/apache/hadoop/commit/2ca9c8d926a8eeb871b2868e6eb4dfb97d7dc63d -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14300) GraphiteSink reports metrics containing undesirable whitespaces
[
https://issues.apache.org/jira/browse/HADOOP-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Claveau updated HADOOP-14300:
Attachment: HADOOP-14300.001.patch
New patch attached with full test coverage (and minor refactoring in the unit
test).
test-patch execution fully succeeded locally.
> GraphiteSink reports metrics containing undesirable whitespaces
> ---
>
> Key: HADOOP-14300
> URL: https://issues.apache.org/jira/browse/HADOOP-14300
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.7.2, 2.7.3
>Reporter: Damien Claveau
>Assignee: Damien Claveau
>Priority: Trivial
> Attachments: HADOOP-14300, HADOOP-14300.001.patch
>
>
> According to the Graphite documentation, the data sent must be in the
> following format:
> But some tag values that are flattened in the metricpath string sometimes
> include whitespaces and violate the carbon/graphite format.
> For example, the dirpath enumeration from the datanode :
> FSDatasetState.org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.Context=FSDatasetState.StorageInfo=FSDataset{dirpath='[/data/1/hdfs/data/current,
> /data/2/hdfs/data/current]'}.Hostname=worker1.Capacity 78436466688 1491986609
> Currently, none of the aggregators available in the Graphite ecosystem can
> handle these malformed metrics. These whitespaces should be replaced by
> underscores.
> I will be happy to attach a small patch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14300) GraphiteSink reports metrics containing undesirable whitespaces
[
https://issues.apache.org/jira/browse/HADOOP-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967325#comment-15967325
]
Damien Claveau commented on HADOOP-14300:
-
A couple of things :
1. There is already a unit test class named
org.apache.hadoop.metrics2.impl.TestGraphiteMetrics
Can I enrich it with my test, or should I rename it TestGraphiteSink to conform
to the naming rules, or should I add a new class TestGraphiteSink ?
2. I think I will have 2 more Jira to open to fully achieve the GraphiteSink.
The first one is a feature addition to filter out tags from the metric prefix
string.
That would be useful to avoid the situation I have with the tag "StorageInfo".
I will propose it like in https://issues.apache.org/jira/browse/HADOOP-7507
The second one will be a fix for the regression described here
https://issues.apache.org/jira/browse/HADOOP-11400?focusedCommentId=14977496=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14977496
Any general advice before I start that work ?
> GraphiteSink reports metrics containing undesirable whitespaces
> ---
>
> Key: HADOOP-14300
> URL: https://issues.apache.org/jira/browse/HADOOP-14300
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.7.2, 2.7.3
>Reporter: Damien Claveau
>Priority: Trivial
> Attachments: HADOOP-14300
>
>
> According to the Graphite documentation, the data sent must be in the
> following format:
> But some tag values that are flattened in the metricpath string sometimes
> include whitespaces and violate the carbon/graphite format.
> For example, the dirpath enumeration from the datanode :
> FSDatasetState.org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.Context=FSDatasetState.StorageInfo=FSDataset{dirpath='[/data/1/hdfs/data/current,
> /data/2/hdfs/data/current]'}.Hostname=worker1.Capacity 78436466688 1491986609
> Currently, none of the aggregators available in the Graphite ecosystem can
> handle these malformed metrics. These whitespaces should be replaced by
> underscores.
> I will be happy to attach a small patch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-14300) GraphiteSink reports metrics containing undesirable whitespaces
[
https://issues.apache.org/jira/browse/HADOOP-14300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Claveau updated HADOOP-14300:
Attachment: HADOOP-14300
Patch attached
> GraphiteSink reports metrics containing undesirable whitespaces
> ---
>
> Key: HADOOP-14300
> URL: https://issues.apache.org/jira/browse/HADOOP-14300
> Project: Hadoop Common
> Issue Type: Bug
> Components: metrics
>Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.7.1, 2.7.2, 2.7.3
>Reporter: Damien Claveau
>Priority: Trivial
> Attachments: HADOOP-14300
>
>
> According to the Graphite documentation, the data sent must be in the
> following format:
> But some tag values that are flattened in the metricpath string sometimes
> include whitespaces and violate the carbon/graphite format.
> For example, the dirpath enumeration from the datanode :
> FSDatasetState.org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.Context=FSDatasetState.StorageInfo=FSDataset{dirpath='[/data/1/hdfs/data/current,
> /data/2/hdfs/data/current]'}.Hostname=worker1.Capacity 78436466688 1491986609
> Currently, none of the aggregators available in the Graphite ecosystem can
> handle these malformed metrics. These whitespaces should be replaced by
> underscores.
> I will be happy to attach a small patch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-14300) GraphiteSink reports metrics containing undesirable whitespaces
Damien Claveau created HADOOP-14300:
---
Summary: GraphiteSink reports metrics containing undesirable
whitespaces
Key: HADOOP-14300
URL: https://issues.apache.org/jira/browse/HADOOP-14300
Project: Hadoop Common
Issue Type: Bug
Components: metrics
Affects Versions: 2.7.3, 2.7.2, 2.7.1, 2.7.0, 2.6.0, 2.5.0
Reporter: Damien Claveau
Priority: Trivial
According to the Graphite documentation, the data sent must be in the following
format:
But some tag values that are flattened in the metricpath string sometimes
include whitespaces and violate the carbon/graphite format.
For example, the dirpath enumeration from the datanode :
FSDatasetState.org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.Context=FSDatasetState.StorageInfo=FSDataset{dirpath='[/data/1/hdfs/data/current,
/data/2/hdfs/data/current]'}.Hostname=worker1.Capacity 78436466688 1491986609
Currently, none of the aggregators available in the Graphite ecosystem can
handle these malformed metrics. These whitespaces should be replaced by
underscores.
I will be happy to attach a small patch.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
