Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/22485#discussion_r220185883
--- Diff:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
---
@@ -168,6 +170,15 @@ protected void serviceInit(Configuration conf) throws
Exception {
TransportConf transportConf = new TransportConf("shuffle", new
HadoopConfigProvider(conf));
blockHandler = new ExternalShuffleBlockHandler(transportConf,
registeredExecutorFile);
+ // register metrics on the block handler into the Node Manager's
metrics system.
+ YarnShuffleServiceMetrics serviceMetrics =
+ new YarnShuffleServiceMetrics(blockHandler.getAllMetrics());
+
+ MetricsSystemImpl metricsSystem = (MetricsSystemImpl)
DefaultMetricsSystem.instance();
+ metricsSystem.register(
+ "shuffleService", "Metrics on the Spark Shuffle Service",
serviceMetrics);
--- End diff --
Probably best to make this sparkshuffleservice as well.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
