[ https://issues.apache.org/jira/browse/HADOOP-10468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061022#comment-14061022 ]
Haohui Mai commented on HADOOP-10468: ------------------------------------- Looking at the code for a while to refresh my memory. Here is a quick recap: # Metric2 implements hierarchical configuration with {{SubsetConfiguration}}. The configuration key has the format {{foo.bar.spam}}, where each parts of the configuration is a hierarchy. For example, {{spam}} is a child of {{bar}}, and {{bar}} is a children of {{foo}. Metric2 turns some parts of the hierarchy keys in lowercase: (MetricsConfig:87) {code} MetricsConfig(Configuration c, String prefix) { super(c, prefix.toLowerCase(Locale.US), "."); } {code} All keys of immediate children will be stored in lowercase strings (MetricConfig:151) {code} Map<String, MetricsConfig> getInstanceConfigs(String type) { Map<String, MetricsConfig> map = Maps.newHashMap(); MetricsConfig sub = subset(type); for (String key : sub.keys()) { Matcher matcher = INSTANCE_REGEX.matcher(key); if (matcher.matches()) { String instance = matcher.group(1); if (!map.containsKey(instance)) { map.put(instance, sub.subset(instance)); } } } return map; } {code} # To look up the value in the hierarchical configuration, {{MetricConfig}} first finds the key in itself and then reconstructs the full key for its parent by concatenating the prefix, the delimiter and the key itself (SubsetConfiguration:88): {code} protected String getParentKey(String key) { if ("".equals(key) || key == null) { return prefix; } else { return delimiter == null ? prefix + key : prefix + delimiter + key; } } {code} In this test case, the reconstructed key is {{test.sink.collector.queue.capacity}} instead of {{test.sink.Collector.queue.capacity}}, which leads to the failure. > TestMetricsSystemImpl.testMultiThreadedPublish fails intermediately > ------------------------------------------------------------------- > > Key: HADOOP-10468 > URL: https://issues.apache.org/jira/browse/HADOOP-10468 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 2.5.0 > Reporter: Haohui Mai > Assignee: Haohui Mai > Priority: Blocker > Attachments: HADOOP-10468.000.patch, HADOOP-10468.001.patch, > HADOOP-10468.2.patch > > > {{TestMetricsSystemImpl.testMultiThreadedPublish}} can fail intermediately > due to the insufficient size of the sink queue: > {code} > 2014-04-06 21:34:55,269 WARN impl.MetricsSinkAdapter > (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full > queue and can't consume the given metrics. > 2014-04-06 21:34:55,270 WARN impl.MetricsSinkAdapter > (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full > queue and can't consume the given metrics. > 2014-04-06 21:34:55,271 WARN impl.MetricsSinkAdapter > (MetricsSinkAdapter.java:putMetricsImmediate(107)) - Collector has a full > queue and can't consume the given metrics. > {code} > The unit test should increase the default queue size to avoid intermediate > failure. -- This message was sent by Atlassian JIRA (v6.2#6252)