jojochuang commented on code in PR #11719:
URL: https://github.com/apache/ozone/pull/11719#discussion_r4364759164
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientMetrics.java:
##########
@@ -79,13 +94,67 @@ public void init() {
}
}
+ /**
+ * Set by each {@link XceiverClientCreator} from its configuration; read
when the metrics are created.
+ */
+ public static void setMetricsMode(MetricsMode mode) {
+ metricsMode = mode;
+ }
+
public static XceiverClientMetrics create() {
- DefaultMetricsSystem.initialize(SOURCE_NAME);
+ if (startsMetricsSystem(metricsMode, classLoader())) {
+ DefaultMetricsSystem.initialize(SOURCE_NAME);
+ }
+ // Registering is cheap without a started metrics system: the source goes
live if the system starts later.
MetricsSystem ms = DefaultMetricsSystem.instance();
return ms.register(SOURCE_NAME, "Storage Container Client Metrics",
new XceiverClientMetrics());
}
+ static boolean startsMetricsSystem(MetricsMode mode, ClassLoader loader) {
+ switch (mode) {
+ case ALWAYS:
+ return true;
+ case NEVER:
+ return false;
+ default:
+ return isSinkConfigured(SOURCE_NAME, loader);
+ }
+ }
+
+ /**
+ * Whether the metrics system would configure a sink for the prefix. Same
lookup as Hadoop's MetricsConfig:
+ * the first of hadoop-metrics2-[prefix].properties and
hadoop-metrics2.properties found on the classpath, with
+ * [prefix].sink.[instance].[option] keys; "*.sink" keys only supply
defaults. Uses java.util.Properties to avoid
+ * loading Commons Configuration.
+ */
+ static boolean isSinkConfigured(String prefix, ClassLoader loader) {
+ final String lowerPrefix = StringUtils.toLowerCase(prefix);
+ for (String name : new String[] {"hadoop-metrics2-" + lowerPrefix +
".properties", "hadoop-metrics2.properties"}) {
+ final URL url = loader.getResource(name);
+ if (url == null) {
+ continue;
+ }
+ final Properties properties = new Properties();
+ try (InputStream in = url.openStream()) {
+ properties.load(in);
Review Comment:
[P2] Detect sinks defined in included configuration files
Hadoop's Commons Configuration parser supports include/includeoptional
directives. If hadoop-metrics2.properties includes another file containing the
sink settings, initialization previously loaded that sink. java.util.Properties
treats include as an ordinary property, so AUTO finds no sink and skips
initialization. Resolve included configurations, or conservatively initialize
when an inclusion directive is present.
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientMetrics.java:
##########
@@ -79,13 +94,67 @@ public void init() {
}
}
+ /**
+ * Set by each {@link XceiverClientCreator} from its configuration; read
when the metrics are created.
+ */
+ public static void setMetricsMode(MetricsMode mode) {
+ metricsMode = mode;
+ }
+
public static XceiverClientMetrics create() {
- DefaultMetricsSystem.initialize(SOURCE_NAME);
+ if (startsMetricsSystem(metricsMode, classLoader())) {
+ DefaultMetricsSystem.initialize(SOURCE_NAME);
+ }
+ // Registering is cheap without a started metrics system: the source goes
live if the system starts later.
MetricsSystem ms = DefaultMetricsSystem.instance();
return ms.register(SOURCE_NAME, "Storage Container Client Metrics",
new XceiverClientMetrics());
}
+ static boolean startsMetricsSystem(MetricsMode mode, ClassLoader loader) {
+ switch (mode) {
+ case ALWAYS:
+ return true;
+ case NEVER:
+ return false;
+ default:
+ return isSinkConfigured(SOURCE_NAME, loader);
+ }
+ }
+
+ /**
+ * Whether the metrics system would configure a sink for the prefix. Same
lookup as Hadoop's MetricsConfig:
+ * the first of hadoop-metrics2-[prefix].properties and
hadoop-metrics2.properties found on the classpath, with
+ * [prefix].sink.[instance].[option] keys; "*.sink" keys only supply
defaults. Uses java.util.Properties to avoid
+ * loading Commons Configuration.
+ */
+ static boolean isSinkConfigured(String prefix, ClassLoader loader) {
+ final String lowerPrefix = StringUtils.toLowerCase(prefix);
+ for (String name : new String[] {"hadoop-metrics2-" + lowerPrefix +
".properties", "hadoop-metrics2.properties"}) {
+ final URL url = loader.getResource(name);
+ if (url == null) {
+ continue;
Review Comment:
[P2] Preserve Hadoop's filesystem configuration lookup
Hadoop's MetricsConfig supports configuration lookup in the working
directory and user home, as well as the classpath. A client with
hadoop-metrics2-xceiverclientmetrics.properties outside its classpath could
previously publish to its configured sink. getResource() misses that file, so
AUTO now skips initialization and disables the sink. Preserve Hadoop's
supported lookup locations and precedence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
