[
https://issues.apache.org/jira/browse/SPARK-25642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718216#comment-16718216
]
ASF GitHub Bot commented on SPARK-25642:
----------------------------------------
vanzin commented on a change in pull request #22498: [SPARK-25642] : Adding two
new metrics to record the number of registered connections as well as the
number of active connections to YARN Shuffle Service
URL: https://github.com/apache/spark/pull/22498#discussion_r240835732
##########
File path:
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
##########
@@ -199,6 +191,18 @@ protected void serviceInit(Configuration conf) throws
Exception {
port = shuffleServer.getPort();
boundPort = port;
String authEnabledString = authEnabled ? "enabled" : "not enabled";
+
+ // register metrics on the block handler into the Node Manager's metrics
system.
+ blockHandler.getAllMetrics().getMetrics().put("numRegisteredConnections",
+ shuffleServer.getRegisteredConnections());
Review comment:
nit: indented too far, here and in others.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add new Metrics in External Shuffle Service to help determine Network
> performance and Connection Handling capabilities of the Shuffle Service
> ---------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-25642
> URL: https://issues.apache.org/jira/browse/SPARK-25642
> Project: Spark
> Issue Type: Improvement
> Components: Shuffle, Spark Core
> Affects Versions: 2.4.0
> Reporter: Parth Gandhi
> Priority: Minor
>
> Recently, the ability to expose the metrics for YARN Shuffle Service was
> added as part of [SPARK-18364|[https://github.com/apache/spark/pull/22485]].
> We need to add some metrics to be able to determine the number of active
> connections as well as open connections to the external shuffle service to
> benchmark network and connection issues on large cluster environments.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
